Extracting structured knowledge from unstructured text

ABSTRACT

Embodiments of the present invention relate to knowledge representation systems which include a knowledge base in which knowledge is represented in a structured, machine-readable format that encodes meaning. Techniques for extracting structured knowledge from unstructured text and for determining the reliability of such extracted knowledge are also described.

1 RELATED APPLICATION DATA

The present application is a continuation of and claims priority under35 U.S.C. 120 to U.S. patent application Ser. No. 13/106,562 forExtracting Structured Knowledge from Unstructured Text filed on May 12,2011, which claims priority under 35 U.S.C. 119(e) to U.S. ProvisionalPatent Application No. 61/334,990 for High Precision InformationExtraction filed May 14, 2010, the entire disclosures of both of whichare incorporated herein by reference for all purposes.

The present application is also related to U.S. Pat. Nos. 7,013,308,7,707,160, 8,666,928, 8,468,122, and 8,838,659, the entire disclosure ofeach of which is incorporated herein by reference for all purposes.

2 BACKGROUND OF THE INVENTION

Currently almost all the real world information that is stored on theinternet is stored within documents: web pages or other files containingnatural language. These documents are held on millions of computers andif linked with hypertext links are done so according to the whims of theindividual authors. The documents are in a large variety of differentformats and written in thousands of different natural languages. Thisinformation is unstructured.

This information is also designed for human eyes. Although naturallanguage understanding has always been a major research area inArtificial Intelligence, computers are not capable of understandingnatural language to any great extent. As a consequence, a human userwanting to find something out using the internet has to first locate adocument that might have the answer and then read it. To locate thedocument, the only practical current technique is keyword searching.

In order to find information using keyword searching the human userfirst hopes that a page/document exists which answers the question,hopes again that it has been indexed by a search engine and then triesto imagine what distinctive words will appear in it. If any of the wordsguessed are wrong or the page has not been indexed by the search enginethey will not find the page. If the combination of words requested iscontained on too many other pages the page may be listed but the humanuser will then have to manually read through hundreds or thousands ofsimilar documents before finding the knowledge required.

In addition there is a certain arbitrariness about the words being used.Searching for general information on a person or product with a unique,distinctive name has a high probability of success, but if the search isfor someone with a common name, or for information on something wherethe name also means something else (searching in English for theJapanese board-game “Go” is a very good example) the search will fail,or an extraordinary amount of extra human effort will be needed tolocate the information. Furthermore, different ways of describing thesame thing mean that several different queries often need to be made orthe search may fail. For example, a search for information on “AbrahamLincoln” is likely to produce a differing list of documents to a searchbased on “President Lincoln” or “Abe Lincoln”.

Certain other types of queries are also extremely hard to answer withkeyword searching. Examples are searching for any type of informationwhich is dynamic. An extreme example would be the local time in aspecific international city. This changes every second, so no web pageindexing technique is going to be able to tell you this information atthe moment of the query. Another example of a dynamic query would be toask what the market capitalization of a company is at the current time.The answer to this depends on the precise share price of the companyinvolved. A further example would be trying to discover the current ageor marital status of a celebrity. Pages containing this information, ifthey were ever true, are only true at the time they were written. Searchengines collect all the documents on the web and have littleunderstanding of which contain out-of-date information. Some of theseissues can be addressed with custom programming for the specific type ofquery at issue (e.g. adding stock quote programming to the search engineand checking for ticker symbols) but keyword indexing documents canprovide no general solution.

Another problem may be that the knowledge is conceptualised in a waythat is different from the way that it is described on the web page. Forexample, if one is trying to locate bi-monthly magazines with a searchengine, one is unlikely to turn up any examples where they are describedas being published “every two months”. Another example would be tryingto find all hotels within two kilometres of a specific geographicallocation. It is extremely unlikely that any description of the hotelwill be expressed in exactly that form so any keyword searching for thiswill fail. i.e. Because search engines don't generally understand theknowledge within a document, they cannot infer new knowledge from whatis said.

Another problem with natural language is that keyword searching islanguage specific. Automatic translation between languages isessentially an unsolved problem in Artificial Intelligence and the stateof the art produces very poor results. As a consequence the web islargely partitioned by the languages used to write the pages. Someonesearching in (say) Hungarian only truly has access to the knowledgestored in that part of the web which is written in the same language.

Even if a document is found that appears to answer the question, theuser may not know how much faith to place in the veracity of what isasserted. The facts asserted within this document may be incorrect orout of date. No general scheme exists on the web for assessing how muchconfidence can be placed in the veracity of any information contained ina web page. The page could contain errors and even the authorship of thedocument may not be clear.

An example of a prior art search-engine interaction illustrating some ofthese problems is shown in FIG. 1. The user has typed a very simplequestion about a popular musician in the search box (102) and the searchengine has responded with a list of documents (104). The web contains avery strong bias towards contemporary people, especially celebrities,and there is no shortage of information on the web which would allow aperfect system to answer this question. In fact there are many thousandsof web pages with information in them suitable for answering it.However, the list of documents bears very little similarity to what isbeing asked and the user would have to experiment further and readthrough a number of documents to get an answer.

The disadvantages of keyword searching are even more extreme when theuser is not human but rather an automated system such as anothercomputer. The software within a website or other automated system needsthe knowledge it requires for its processing in a form it can process.In almost all cases, documents found with keyword searching are notsufficiently processable to provide what is needed. As a consequencealmost all the world's computer systems have all the knowledge they needstored in a local database in a local format. For example, automatedscheduling systems wanting to know whether a particular date is anational holiday access a custom written routine to provide thisinformation, they do not simply consult the internet to find out theanswer.

Knowledge in structured form is knowledge stored in a form designed tobe directly processable to a computer. It is designed to be read andprocessed automatically. Structured form means that it is not stored asnatural language. It is knowledge stored in a pre-determined formatreadable and processable by the computer. Knowledge in structured formwill include identifiers which denote objects in the real world andexamples will include assertions of information about these identifiedobjects. An example of such an assertion would be the assertion that anidentified relationship exists between two or more identified objects orthat a named attribute applies to an identified object. (Individualinstances of structured knowledge are referred to herein as “facts” or“assertions”.)

To fully understand the potential advantages of embodiments of thepresent invention it is also important to understand some issuesrelating to the broadness or narrowness of the domain of knowledge beingrepresented. Knowledge stored in (say) a company's employee relationaldatabase may be in structured form but is in an extremely narrow domain.The representation is entirely local and only meets the needs of thenarrow computer application which accesses it. Typically data stored ina computer system is designed to be used by, and can only be fullyexploited by, the software within that system. In contrast, generalknowledge is knowledge falling within an extremely wide domain. Generalknowledge stored in structured form represents general knowledge in sucha way that it combines at least some of the universal meaningfulnessadvantages of natural language with the machine-processing advantages ofother computer data. However, there are very significant difficulties toovercome to achieve this.

General knowledge in structured form has a variety of uses by acomputer, including direct answering of natural language questions, andassistance with other forms of natural language processing (such asmining data from documents). It can even assist with keyword searching.For example, with the example above, if the structural knowledge existsthat the strings “Abe Lincoln” and “President Abraham Lincoln” bothdenote the same unique entity a search engine using such a knowledgebase could return documents containing either term when only one wasentered by the user.

Building a large database of general structured knowledge presentsserious difficulties. There are considerable difficulties in designing aknowledge representation method that is sufficiently expressive torepresent a wide range of knowledge yet also sufficiently elementary inform to allow effective automated processing (such as inference andquery responses). Building a knowledge base by hand (i.e. using directhuman interaction as the source of the knowledge) is slow, so to buildthe largest possible knowledge base in a reasonable time requires alarge number of people contributing possibly combined with automatedmethods.

One way to enable people to contribute is to select, hire and trainsalaried staff and then pay them to add this knowledge. Training themwould typically require educating them about the underlying knowledgerepresentation syntax and teaching them about what is already in theknowledge base.

However, to open up the process to the largest number of people (such asgeneral users of the internet) requires enabling access to at least someof the knowledge addition process to untrained users.

Enabling untrained users to add general knowledge in structured form toa knowledge base presents a number of very significant problems.

First, these users are unlikely to know anything of the underlyingknowledge representation technology so if untrained users are genuinelyto be used, they will ideally need to be able to assert facts in a waythat is natural to them and distinct from the knowledge representationformat.

Secondly, these users are untrusted and potentially malicious. For thisreason it isn't desirable to simply permanently add all knowledgeasserted by such users to the published knowledge base. Desirablymethods are needed to distinguish between true and untrue facts and toretain true facts while removing (or never publishing) untrue facts.

Thirdly, adding knowledge should desirably not require any previousknowledge of what is already in the knowledge base. If prior familiaritywith the ontology or other facts that are already in the knowledge baseis required, untrained users will find it more difficult to addknowledge.

All of the above issues both with knowledge representation generally andwith the knowledge addition process are directly addressed in variousembodiments of the present invention.

3 SUMMARY OF THE INVENTION

Embodiments of the present invention relate to repositories of generalknowledge, stored in structured form. According to one class ofembodiments, methods, systems, and computer program products areprovided for extracting structured knowledge from unstructured text foruse in a knowledge representation system. The knowledge representationsystem includes a knowledge base that represents knowledge using astructured, machine-readable format. The structured, machine-readableformat includes fact triples. Each fact triple includes two entityobjects and a relationship object expressing a relationship between thetwo entity objects. Sentences are identified in the unstructured text.Each of a subset of the sentences is converted to one or more simplifiedassertion statements of the form: subject noun phrase, verb phrase,object noun phrase. Each of a subset of the simplified assertionstatements is converted to a corresponding fact triple. A reliability isdetermined for each of the fact triples.

According to another class of embodiments, methods, systems, andcomputer program products are provided for extracting structuredknowledge from unstructured text for use in a knowledge representationsystem. The knowledge representation system includes a knowledge basethat represents knowledge using a structured, machine-readable format.The structured, machine-readable format includes fact triples. Each facttriple includes two entity objects and a relationship object expressinga relationship between the two entity objects. A question and acorresponding answer are identified in the unstructured text. Thequestion is at least partially translated. Additional information isextracted from the answer with reference to the partially translatedquestion. One of the fact triples is generated using the at leastpartially translated question and the additional information extractedfrom the answer.

According to one specific class of embodiments in which the question isa yes-or-no question, at least partially translating the questionincludes translating the question to an intermediate fact triple, andgenerating one of the fact triples includes determining a polarity forthe intermediate fact triple with reference to the additionalinformation extracted from the answer.

According to another specific class of embodiments, at least partiallytranslating the question includes translating the question to anintermediate fact triple in which one of the two corresponding entityobjects includes an unknown entity object, and generating one of thefact triples includes replacing the unknown entity object in theintermediate fact triple with a known entity object determined from theadditional information extracted from the answer.

According to yet another specific class of embodiments in which thequestion requires a complex response not suited for representation inthe knowledge base, one or more entities identified in the partiallytranslated question and referenced in the answer are disambiguated.

A further understanding of the nature and advantages of embodiments ofthe present invention may be realized by reference to the remainingportions of the specification and the drawings.

4 BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 gives an example of a prior art search with a search-engine. Aquestion has been turned into a list of documents based on themcontaining similar words.

FIG. 2 shows an embodiment of the present invention “plugged into” thesame search engine and responding to the same question using structuredknowledge. A perfect answer is provided to the user and the list ofdocuments is relegated to serving as supplementary information.

FIG. 3 illustrates components in the preferred embodiment of theinvention.

FIG. 4 shows a method for answering a query with “no” instead of“unknown”

FIG. 5 shows how knowledge about the completeness of the resultsreturned can be given in query processing.

FIG. 6 shows how queries are processed in one embodiment.

FIG. 7 shows a question answered with multiple answers and completenessinformation provided.

FIG. 8 shows a question answered with both a concise and a detailedexplanation.

FIG. 9 shows a method for translating a question or fact assertion fromnatural language into internal form.

FIG. 10 shows a method for eliminating improbable candidate translationsusing semantic constraint knowledge.

FIG. 11 shows how multiple translation candidates are dealt with moregenerally.

FIG. 12 shows two example questions with ambiguity being dealt with.

FIG. 13 illustrates the profile system with four different profilesbeing given for the same entity.

FIG. 14 illustrates the profile showing system specific data.

FIG. 15 shows a method for selecting a default profile template for agiven object.

FIG. 16 shows a method for turning a profile template and object into aprofile.

FIG. 17 shows part of a profile template being processed.

FIG. 18 shows part of a profile template containing iterator nodes beingprocessed.

FIG. 19 shows a method of authenticating a user using their real worldidentity.

FIG. 20 shows a method of selecting an object.

FIG. 21 shows a method of allowing a user to add a new (non class, nonrelation) object.

FIG. 22 illustrates an exemplary interaction with a user adding a newobject.

FIG. 23 is a continuation of FIG. 22.

FIG. 24 shows a method of allowing a user to add a new class.

FIG. 25 illustrates an exemplary interaction with a user adding a newclass.

FIG. 26 is a continuation of FIG. 26.

FIG. 27 shows a method of allowing a user to add a new relation.

FIG. 28 illustrates an exemplary interaction with a user adding a newrelation.

FIG. 29 is a continuation of FIG. 28.

FIG. 30 is a continuation of FIG. 29.

FIG. 31 is a continuation of FIG. 30.

FIG. 32 shows a method of dealing with a sequence of facts collected forassertion by a process.

FIG. 33 shows a method of collecting denotational strings for a newobject.

FIG. 34 shows a method of allowing a user to add a new fact to thestatic knowledge base.

FIG. 35 illustrates a user adding a new fact where all but one elementhas been pre-specified.

FIG. 36 shows a method for collecting essential facts from a user abouta newly added object.

FIG. 37 shows a method for collecting temporal data from a userpertaining to a transient fact.

FIG. 38 shows a method for collecting source information about a factfrom a user.

FIG. 39 shows a method usable in the user assessment subsystem forcollecting endorsements or contradictions of a fact from a user.

FIG. 40 shows a method usable in the system assessment subsystem forautomatically calculating various types of state information about afact.

FIG. 41 illustrates an exemplary interaction with a user where userassessment and system assessment methods allow an incorrect fact to beremoved from the static knowledge base and the correct version to bepublished.

FIG. 42 illustrates an exemplary interaction with a user where theuser's attempts to abusively assert knowledge are thwarted by twodifferent abuse prevention techniques.

FIG. 43 shows a method of utilising a prior art search engine incombination with an embodiment of the current invention to process auser search query.

FIG. 44 shows a method of enhancing a user search query using knowledgeobtainable from an embodiment of the present invention.

FIG. 45 shows a method of using knowledge about a vertical to helpdisambiguate a question.

FIG. 46 shows a method for users to create vertical question answeringsites of their own choosing.

FIG. 47 shows a method for populating a knowledge base with additionaltrue facts inferred from others.

FIG. 48 shows an example of a system configured to extract facts fromunstructured text according to a specific embodiment of the invention.

FIG. 49 is an example of a representation of a parse tree.

FIGS. 50 and 51 show examples of a quarantine for filtering factsaccording to a specific embodiment of the invention.

5 DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Reference will now be made in detail to specific embodiments of theinvention including the best modes contemplated by the inventors forcarrying out the invention. Examples of these specific embodiments areillustrated in the accompanying drawings. While the invention isdescribed in conjunction with these specific embodiments, it will beunderstood that it is not intended to limit the invention to thedescribed embodiments. On the contrary, it is intended to coveralternatives, modifications, and equivalents as may be included withinthe spirit and scope of the invention as defined by the appended claims.In the following description, specific details are set forth in order toprovide a thorough understanding of the present invention. The presentinvention may be practiced without some or all of these specificdetails. In addition, well known features may not have been described indetail to avoid unnecessarily obscuring the invention.

The structured knowledge representation employed by specific embodimentsof the invention uses primarily a collection of assertions of namedrelationships between pairs of named entities. Each assertion (alsoreferred to herein as a “fact”) is also a named entity and temporal dataabout when a fact is true can be asserted using similar assertions. Thepreferred embodiment supports “negative facts”: assertions of arelationship not being true and “parametered objects” where entities areidentified by a combination of a class with one or more other namedentities. There is also a strong emphasis on natural language factsassociating strings with each object via a relationship whose semanticscorresponds to a natural language concept. These facts facilitate humaninteraction with an embodiment of the invention.

The structured knowledge representation described herein is advantageousin that it allows representation of knowledge of an extremely broadclass. That is, it is operable to represent any entity (including binaryrelationship and attributes) which can be denoted in natural language,i.e., if you can give it a name you can add it to the knowledge basewith a unique recognition string which gives that entity meaning tohumans. The structured knowledge representation is also operable torepresent the presence or absence of any relationship between two ormore such entities, and whether or not a particular attribute applies toa specific entity. The structured knowledge representation is alsooperable to represent points in time when these relationships are valid.

By contrast, in the typical hard-coded, database-driven application, theinformation represented and manipulated is of an extremely narrowdomain. For such applications the developer typically creates a schemaof database tables to store the entities and the relationships betweenentities that the application needs. The developer then hard-codes aprogram that manipulates the data in these tables, e.g., using SQL.

The knowledge domain associated with such applications is extremelynarrow because nothing that happens after the application is launchedever extends this schema beyond the scope for which it was originallydesigned. Users may add data to the tables, but they can never extendwhat can be represented or what kinds of queries can be made.

By contrast, and as will be described, knowledge representation systemsenabled by the present invention can enable users to almost arbitrarilyextend the scope of the knowledge being represented. In fact, the scopeof knowledge represented may be extended every time a new class,attribute or relationship is added.

According to specific embodiments of the invention, queries and queryanswering are also supported. Queries are a machine-readable analogue toa question or knowledge request designed to elicit knowledge from thesystem. In the preferred embodiment, the query answering system cananswer queries with a list of objects that match the query and cananswer “truth queries” (the query analogue to a yes/no question) with“yes”, “no” and “unknown” responses. In some cases “completenessinformation” (whether the list of responses contains all the possibleresponses) can be provided when the query requests a list of entities.

As there are far more facts than can be stored statically, the preferredembodiment also supports knowledge generation. Knowledge generationenables facts to be generated by the system which are not present in thestatic knowledge base. This can be achieved by inference from the factsin the static knowledge base. The knowledge generation system can alsogenerate facts sourced from a third-party database or dynamic sourcesuch as (for example) financial information.

Knowledge generation is implemented in the preferred embodiment via acollection of “generators” which comprise a pattern of the facts whichthey can generate in combination with one or more mechanisms to generatefacts which match this pattern. Some generators achieve this byproviding a query linked to the pattern which if answered providesvalues for unknowns in the pattern thus enabling the generation of thefacts (“dumb generators”). Other generators use some executable codepossibly in combination with a query to generate facts matching thepattern (“smart generators”). Smart generators can be used to generatefacts sourced from an external source or database by accessing thisexternal source and converting the knowledge so retrieved into factsmatching its pattern. Smart generators can also be used to do inferencewhere at least one calculation step is needed to generate the new facts.

Some embodiments additionally support a form of knowledge generationthat takes place prior to a query being executed and results inadditional facts being stored in the knowledge base.

Various embodiments also support the creation of detailed naturallanguage explanations of how a query was answered. The preferredembodiment additionally supports a summarised concise explanationshowing only the facts in the static knowledge base (or an essentialsubset thereof) that were used to respond to the query.

The preferred embodiment also supports question translation. This is thecapability to translate natural language questions or knowledge requestsprovided by a user into a query. In combination with the query answeringsystem this enables internet users to type a natural language questiondirectly into the system and obtain an answer directly. Variousembodiments also support ambiguity resolution by elimination ofimprobable interpretations of the question.

Various embodiments also support the retranslation of a query back intounambiguous natural language. In combination with the questiontranslation system, this enables the user to have confidence that theirquestion has been correctly understood. If the question translationsystem determines that the user's question is ambiguous it also enablesit to present the list of interpretations of their question forselection of the user's intended query.

In addition to use by human users, various embodiments also support useby remote automated systems. In the preferred embodiment a number ofservices are provided including responding to queries. As the queriesand response are in structured form, this service can be of genuine useby a remote non-human user in a way that a traditionaldocument-returning search-engine cannot.

Knowledge addition in the preferred embodiment is achieved by a numberof “processes” which interact with general internet users via a sequenceof web pages containing prompts, text input boxes and buttons. Theseprocesses receive, check and refine the answers provided by users andinclude confirmation pages. Processes can also call other processes assub-processes (which can in turn call additional processes etc.)creating intervening additional sequences of pages within the parentprocess. For example, when a user adds a new entity to the knowledgebase and asserts that this entity belong to a class which is also not inthe knowledge base, the process for adding the class can be immediatelyimplemented returning the user to the initial process (with the class soadded) when it is finished. The calling parent process receives theclass name exactly as if it was an existing class which had beenselected by the user.

In the preferred embodiment the knowledge addition system comprisesprocesses for adding new classes, new relations and new entities ofother types.

Users can also assert new facts which in the preferred embodiment areassertions of a named relationship between two entities and anyassociated temporal information.

The preferred embodiment also has support for natural languagetranslation of facts asserted by users whereby a natural languagesentence can be translated into a combination of one or more facts usinga method similar to the translation of questions and, afterconfirmation, this knowledge added to the static knowledge base.Prompting for the two objects and the named relationship individually isused as a fall-back if the entire assertion cannot be understood.

Various embodiments also support “user assessment” where users canendorse or contradict facts in the static knowledge base and theseassessments are used to remove or hide untrue facts. In the preferredembodiment links to endorse or contradict a fact are provided next tofacts in the static knowledge base displayed to the user. For example,this occurs when presenting the summary explanation generated inresponse to a question or knowledge request provided by a user. When agreat deal of confidence has been gained in the veracity of a fact thepreferred embodiment ceases to accept user assessment on it.

In the preferred embodiment users can authenticate themselves with theid of class [human being] that corresponds to their real identity. Thepreferred embodiment additionally contains mechanisms for users toestablish that they have not appropriated the identity of someone otherthan themselves. In the preferred embodiment, knowledge addition anduser assessment are associated with the user's true identity as thereporter, thereby giving a clear record of the provenance of theknowledge.

Various embodiments also contain a “system assessment” componentoperable to assess the veracity of facts based at least on theirsemantic interaction with other facts in the knowledge base. In thepreferred embodiment facts can be labelled as “contradicted” (insemantic conflict with other facts in the static knowledge base) and“superfluous” (believed true but which can already be generated by thesystem). System assessment is done on all newly added facts to thestatic knowledge base and the user who has added a fact that iscontradicted by other facts in the static knowledge base is given anopportunity to use user assessment to draw attention to and potentiallychange the status of any of those facts which they believe to be untrue.In the preferred embodiment, system assessment can be used toresuscitate facts previously thought to be untrue when for example, oneor all of the facts in conflict with the newly added fact is laterreassessed (via user assessment or otherwise) as untrue. Otherembodiments may use system assessment to prevent untrue facts from beingadded to the system at all.

Various embodiments also support additional mechanisms for preventingthe addition of untrue facts by mistaken or abusive users including theability to block certain patterns of facts from being added and rankingof users based on their track record of adding highly quality knowledge.More trust is associated with the users of higher rank, more weightgiven to the facts they assert and more weight to their user assessmentsresulting in a higher probability of publication.

Various embodiments also support the generation of “profiles” givinggeneral information about a particular entity based on its class and theknowledge about that entity in the system. This is implemented in thepreferred embodiment via a collection of profile templates which definethe contents of an information screen and what queries need to be run topopulate it. The preferred embodiment supports one or more differentprofiles being supported for a particular class giving a differentemphasis to the object being profiled. It is also possible to navigatethrough the classes that an object is a member of, giving a profiletailored to that class for the same entity. In the preferred embodiment,where information is missing on a profile, links can be providedenabling a user to add the missing knowledge with only the missingknowledge being prompted for.

As the underlying representation of knowledge is in a form separate fromnatural language, various embodiments support user interactions with thesystem via multiple natural languages and with the users using differentnatural languages sharing access to at least some of the structuredknowledge.

Various embodiments additionally support the representation of namedvertical areas to which facts and queries can be classified. Theseembodiments enable more effective understanding of the user's intent incertain circumstances and also allows the creation of vertical websiteswhich are focussed on a particular area of knowledge.

In certain embodiments users can automatically create their own verticalversion of a question answering website.

Various embodiments also comprise a search-engine component operable toproduce a list of documents (e.g. web pages) ordered by relevance to aquery entered by a user. This component can be used to produce resultsin addition to the normal response to a user's question or as afall-back when the question has not been successfully translated or thesystem cannot respond to the query.

In a related embodiment the present invention is implemented as a“plug-in” to a pre-existing search engine. The search-engine queryentered by the user is processed by both the search-engine to produce alist of documents and by this embodiment to possibly produce a resultoriginating from the structured-knowledge source. A successful responsefrom the plug-in is presented above the search-engine results. Ifunsuccessful, the standard search-engine output is presented to the userand the user is no worse off than they would have been without theplug-in.

A user interaction with this plug-in embodiment is illustrated in FIG.2. The user question or knowledge request (202) has been passed boththrough the search-engine search to produce a list of documents andadditionally through an embodiment of this invention. The questiontranslation component has received the user question and produced aquery. The query answering system has then processed this query usingknowledge generation and references to structured knowledge facts in thestatic knowledge base, producing an answer which is translated back intonatural language for presentation to the user (204). The query answeringsystem has also produced a concise explanation for the answer bypresenting the facts in the static knowledge base which were used toanswer this query (206). (The needed generated facts are not shown.) Oneof the facts used to answer the question can be confirmed orcontradicted by the user (207) via the user assessment system. Adetailed explanation including the generated facts and the steps takento generate them was also produced, accessible to the user via a link(212). This embodiment has also retranslated the query back intounambiguous natural language to demonstrate that the user's question hasbeen understood (208). The prior art list of web pages is still producedbut has now been relegated to supplementary information (210).

The following sections describe in detail how embodiments of the presentinvention work. It should be noted that many of the choices made indescribing the preferred embodiment are fairly arbitrary, including, forexample, the choice of ontology, the syntax, and the names given to theclasses and objects used as examples. A variety of different choiceswill be obvious from the principles described herein.

Moreover, much of the description of the processing of the Englishlanguage is adaptable to other specific languages and thus thedescription should not be taken as limiting the choice of naturallanguage to English. Indeed, as the underlying knowledge is representedin a form separate from natural language, supporting multiple naturallanguages on the same system is a desirable feature of variousembodiments. (See section 5.6.5 for more on this.)

Another rich source of additional embodiments is the precise way thatthe various processes and components have been subdivided. Thesubdivision of a complex system is sound engineering practice andenables reuse of the components and clearer thinking about the highlevel method. However, different ways of dividing what is described willbe obvious.

As the internet is where the primary commercial opportunities for thepresent invention are currently believed to lie, all the examplescontained herein assume an internet-based embodiment. However, what isdescribed is readily adaptable to operate on other computer networks andexample internet-based embodiments should not be read as precludingother forms of network. For example, an embodiment based on a mobilephone network where the client machines are user handsets should bereadily apparent to anyone reading this description.

Similarly, all the examples contained herein show a web-basedinteraction with users. Again, what is described is readily adaptable toother forms of user-interface/user interaction. For example, theinteraction with users could be implemented using a largely text-basedexchange of commands and responses.

Many further choices for implementing much of the technology areavailable, and many different embodiments will be obvious to anyone ofordinary skill in the art who has read what follows. Any specificsdescribed should not be interpreted as limiting the generality of whatis claimed.

5.1 Components

FIG. 3 shows some of the components in the preferred embodiment. (Manyof these components are optional and simply add to the overallfunctionality/utility of the system. They may not be present in otherembodiments.)

One or more client computers (302) with a human user (303) can accessthe system via a web-interface (310) on at least one server (308).

Additionally, one or more remote computers making automated queries(306) can access the system via a remote computer interface (312). Theremote computer interface is described in section 5.17.

The underlying knowledge is stored in one or more static knowledge bases(318). The static knowledge base is described in section 5.2 and thepreferred embodiment knowledge representation method used to representthe knowledge stored in the static knowledge is described in section 5.3

Knowledge can be added to the static knowledge base by users using theknowledge addition subsystem (326). This component and its subcomponentsare described in section 5.10.

Users are also able to correct and endorse added knowledge via the userassessment component (334). This is described in section 5.10.17

The system is also able to analyse and label facts using systemassessment (316). This is described in section 5.11

Natural language translation (324) enables translation between naturallanguage and internal representations. e.g. It can translate a naturallanguage question into a query and natural language assertions ofknowledge into one or more corresponding facts. Translation of questionsis described in section 5.6.6, translation of factual assertions isdescribed in section 5.6.10). Both these components are implemented inthe preferred embodiment by referring to a store of translationtemplates (325). These provide a pre-determined pattern for matchingagainst natural language strings and further data enabling naturallanguage strings matching the pattern to be converted to the internalrepresentation.

Query processing (314) enables the retrieval of knowledge from thesystem. Queries may be the output of the natural language translationsystem (324) or provided by remote computers (306). Query processing isdescribed in section 5.5

The knowledge generation subsystem (320) provides facts which are notpresent in the static knowledge base often by inferring new knowledgefrom the facts that are present in the static knowledge base. Thepreferred embodiment uses a store of generators (322) which describepatterns of fact which they are capable of generating along with one ormore mechanisms to generate these facts. Such a mechanism can be just aquery (a dumb generator), or some program code optionally in combinationwith a query (a smart generator). Knowledge generation is described insection 5.4

The profile generation system (330) enables the creation of a collectionof information about a particular object. In the preferred embodimentthis is a web page. In the preferred embodiment profile generation isachieved by use of a store of profile templates (332) which specify theknowledge to be displayed, its format and how to obtain it.

User authentication is achieved via the authentication subsystem (328).In the preferred embodiment users can authenticate themselves usingtheir real-world identity. This is discussed in section 5.9.

5.2 The Static Knowledge Base

As used herein, a “static knowledge base” is the term for acomputer-accessible persistent store comprising knowledge represented instructured form. A persistent store could be a memory or memories of anytype capable of holding the knowledge long term. For speed, variousembodiments may hold the data in a long term store but temporarily cacheit in a fast non-persistent memory such as RAM for access by othercomponents of the system.

In the preferred embodiment the static knowledge base is a collection offacts represented using the knowledge representation method of thepreferred embodiment described below, stored in one or more relationaldatabases on one or more server computers.

5.3 Knowledge Representation

Knowledge representation is the methodology by which knowledge instructured form is represented within at least the static knowledgebase.

Methods of representing knowledge in structured form include:

(a) Semantic nets (graph-like representations where the nodes correspondto objects and the edges to relationships);

(b) Logic, a machine-readable mathematical language of pre-determinedsyntax used to represent the knowledge. Logics are substantially simplerand more rigorously defined than natural language. Types of logicinclude predicate logic and propositional logic.(c) Frames. Frames represent objects as a set of slots (attributes) andassociated values.

Embodiments of the current invention can contain a static knowledge basecontaining facts using at least one alternative structured knowledgerepresentation. However, the preferred embodiment uses primarily acombination of simple assertions asserting a named relationship betweentwo objects to represent knowledge. The relation can be negative andcertain objects can comprise one or more further objects (“parameteredobjects”). Each fact is also an object allowing facts to make assertionsabout other facts.

A detailed description of the knowledge representation used in thepreferred embodiment is now disclosed:

5.3.1 Objects

Objects are individual entities. They can include physical objects inthe real world (individual people, places, buildings etc.), conceptualobjects (numbers, organisations etc.), attributes, quantities, classesetc.

All identified objects have a unique id within the system. This namemust be unique to identify the object and in the preferred embodimentshould correspond to a common, yet fairly specific natural language nounor noun phrase for the same object (for relations, see section 5.3.3, apresent tense central form is used). Instances are usually given theproper name for the object if there is one. If the proper name is notunique then a noun phrase is used including the proper name. In thepreferred embodiment these names can include spaces making them veryclose to natural language.

As a last resort, in some embodiments, one can distinguish betweenobjects that would otherwise have been assigned the same name, by addingan integer at the end (after a space). If the integer is missed off itis assumed to be 1 so new objects which clash with existing objectsstart their numbering with 2. For example:

e.g. [california] is the object name for the US state of California.[william jefferson clinton] is the object name for the former USpresident.

Note how the names are written in square brackets in the preferredembodiment. This identifies the thing as an id and also means that suchnames are recognizable as being part of the preferred embodiment of thepresent invention, even when written out in some context where what itis would not otherwise be obvious. For example, a person could printtheir id in square brackets on business cards and their name would bereadable both as a natural language name and recognisable as an id forthe embodiments of present invention. People with the business cardcould then use the id to find out more information about the holder.

5.3.1.1 String Objects

Generally the above syntax is unrelated to the ontology used or theclass of the object. One exception in the preferred embodiment is theclass of strings (sequences of characters). Instances of this class aresimply the string itself put in quotes, e.g. [“William”] is the name forthe sequence of characters ‘W’..‘i’.. ‘l’.. ‘l’.. ‘i’.. ‘a’..‘m’—itmeans nothing more than that. Such objects are useful for statinginformation used for translation and for parametered objects.

One common use of strings is to indicate denotational strings. As usedherein denotational strings are strings which are used in naturallanguage to denote an object in the system. For example, the strings“Abe Lincoln”, “Abraham Lincoln” and “President Lincoln” aredenotational strings for former US president Abraham Lincoln; “green” isa denotational string for the attribute green, etc. Denotational stringscan also denote objects of all types including relations, classes etc.

5.3.1.2 Parametered Objects

Some classes contain an infinite (or extremely large) number of objectsthat can be consistently understood in some way. We can choose to denotesuch objects by a combination of the class name and data. The syntax ofa parametered object in the preferred embodiment is:

[<class name>: [object 1]; [object 2]; . . . ; [object n]]

Parametered objects have at least one object within the name as aparameter. The number of parameters can be fixed for a particular class,e.g. timepoint (a moment in time), or vary, e.g. group (a collection ofobjects regarded as a single object).

For some objects, strings containing the important information are usedas the parameter or parameters. This is especially useful where there isalready a well-established “real-world” syntax for members of the class.A simple example is the class of integers, e.g. [integer: [“8128”]].Integers already have a universal syntax and meaning using the digits0-9 in sequence and the decimal system. It is thus desirable to denotethem using a single string object as the parameter. A more complicatedbut equally valid example is a chess position where a standard way ofdenoting it as strings (and including all the other information such asthe side to move and castling rights) has already been established, e.g.[chess position: [“R7/5p1p/5Kp1/8k6P/p1r5/2P5/8 b--”]].

Another common class of parametered objects used in the preferredembodiment is the timepoint class. Here a single string object is usedwith a format that is not widely used. It is a sequence of integersseparated by “/” characters, denoting (in order), the year, the month,the day, the hour in 24-hour clock, the minute, and the second. Anyfurther integers are tenths, hundredths, thousandths of seconds, etc.,e.g.

[timepoint: [“1999/6/3/15/0”]] is 3 pm on the 3rd of June 1999 UTC. Theaccuracy of this timepoint is within one minute. [timepoint: [“1999”]]specifies a “moment” of time but the accuracy is one year.

Parametered objects are compared by comparing each parameter in turn. Ifthe nature of the class means that order is unimportant (e.g. group) theparameters need to be considered in a pre-determined order (e.g.alphabetical) so that the same objects will be compared as equal.

In the preferred embodiment, parametered objects can also have otherparametered objects as parameters. This nested nature of parameteredobjects can be extended indefinitely deeply. For example, we coulddefine a class “pair” specifically for objects consisting of exactly twothings, e.g. [pair: [integer: [“5”]; [integer: [“7”]] and having done sothere is no reason why we could not create a name for the objectconsisting of the group of this object and other things, e.g. [group:[pair: [integer: [“5”]; [integer: [′7′]]; [abraham lincoln]] is theobject which involves considering the pair (5,7) and Abraham Lincoln asa single thing.

5.3.2 Unique Recognition Data

As used herein unique recognition data is data associated with an objectwhich has the following properties:

It should be perceivable by human users of the system.

It should uniquely distinguish the object from others which might bemistaken for it by the users of the system (e.g. other objects withsimilar names).

It should be generally appreciable. i.e. enough of the data should havemeaning to uniquely distinguish the object to all users (or almost allusers) who may wish to communicate with the embodiment about the object.

For example, someone's name plus their passport number would beperceivable (people can read names and passport numbers). It would alsouniquely distinguish that person from all other people (passport numbersare unique). However, it would not be generally appreciable in mostcircumstances if that person's name was common as most people do notknow other people's passport numbers. As it is not generallyappreciable, it would not count as unique recognition data. However, thename of a person, a collection of common details about them and aphotograph probably would count, as most people wishing to identify thatperson are likely to be able to pick out enough detail from the data touniquely identify that person from anyone else it might be, even if someof the data was not known to them.

5.3.2.1 Unique Recognition Strings

A unique recognition string is unique recognition data coded as asequence of printable characters, readable and understandable by a humanuser.

In the preferred embodiment, objects are associated with a uniquerecognition string.

This association is done with a simple fact using the relation [uniquelytranslates as] (see section 5.3.6 for how facts are asserted). This factmight be generated (see section 5.4).

The purpose of this string is to both uniquely distinguish the objectfrom all other objects which may have similar names and to do so in amanner which allows this to happen in the minds of all (or almost all)the human users who may see this string and who have some familiaritywith the object.

For a person who is very famous for one thing, such a string can oftenbe formed by simply saying their name and this famous thing. Forexample, for [william jefferson clinton], “Bill Clinton, the 43rdPresident of the United States” would be sufficient as it is extremelyunlikely that anyone trying to reference that individual would fail toknow that he was once US president. However, for an individual who isless famous several pieces of information may need to be combined sothat different groups of people have enough information to be sure thatthey are the person being referred to (the “generally appreciated”requirement). For example, “James R. MacDonald, software developer, dateof birth 3rd of April 1975, resident in Cambridge England and employedby Ficton Engineering Ltd” may be sufficient for a non-famous person aseven people who not very familiar with that individual will probably seeenough of what they know to make an identification.

5.3.2.2 Images

In some embodiments, an identifying image may be part of the uniquerecognition data. In cases where everyone who wishes to communicateabout the object has seen it (or knows what it looks like), it may bethe only unique recognition data.

5.3.2.3 Collection of Stored Facts

Other embodiments may use a collection of stored knowledge about theobject together as unique recognition data. Embodiments can offer thisvia a profile of the object (see section 5.7). For example, anembodiment could display the id for the object linked to a profile forthe object. If the user didn't recognise the id, they could click on thelink to see the profile and use this information collected together torecognise the object.

5.3.3 Relations

Relations are things which link together objects. The preferredembodiment uses relationships between two objects. Relationships canexist between physical objects and also between physical objects andnon-physical objects (concepts), e.g. “John is married to Sarah” is anatural language assertion about a relationship between two physicalobjects (in this case people). “The apple is green” asserts arelationship between the attribute “green” with the instance of applebeing talked about. “The book is about Albert Einstein's career” assertsa relationship between a book and the concept of Albert Einstein's workhistory. “The soup tastes salty” asserts a relationship between theattribute “salty” with the soup. All of these natural languageassertions also contain information about time (tense): this will bedealt with below.

In the preferred embodiment, relationships are also objects. Forexample:

[is married to] is the object (relation) that corresponds to the Westernconcept of marriage between a man and woman, i.e. a formalisedmonogamous marriage.

[is an instance of] relates an instance object to a class object, e.g.the relationship between Albert Einstein and the class [human being].

[applies to] relates an attribute object to another object, i.e. it saysthat a certain property applies to something. This second object can beanything: an instance, a class, a relation or even another attribute.

[is a subclass of] relates one class to another and says that the firstclass is a more specific class than the second and that all objects thatare members of the first class are also members of the second. Forexample, this relationship applies between the class [apple] and theclass [fruit].

In the preferred embodiment, relations are typically named by finding apresent tense verb phrase that unambiguously describes the relationship.

5.3.4 Classes In the preferred embodiment all objects are members of atleast one class. Classes define objects with similar characteristics.Class information is thus useful for generation and profile screens (seesection 5.7). An object is related to a class of which it is a member bythe [is an instance of] relation.5.3.4.1 Class Structure

Classes are related by the relation [is a subclass of], so if B is asubclass of A then all objects which are members of B are also membersof A. For example all members of [human being] are members of [livingthing] because [human being] is a subclass of [living thing].

According to various embodiments, classes can also partially overlap.For example, a class could be defined of male living things which wouldbe a subclass of [living thing] with the attribute [male]. However,members would include male human beings as well as male animals whilefemale human beings would be excluded. Another example would be theclass of (say) [blonde person] and [woman]. Classes with no member incommon have the relation [is a distinct class from]

As there is a class of [object] in the preferred embodiment whichincludes everything, the classes in the knowledge base can be considereda tree with the [object] class as the root.

5.3.4.2 Permanent Classes

A permanent class is one where membership by an object cannot change astime goes by. The object is a member of that class for the entiretimeline, i.e. the properties of the class are so core to objects withinit, that is reasonable to say that the object would cease to be thatobject (i.e. a different identifier would be needed) if those propertieswere ever to change. An example of a permanent class would be [tree].Any object which is a tree is always a tree and if something radicalwere to be done to it to make it not a tree, such as cutting it down andturning it into a table, it is reasonable to think of the new object asa different object with a different identifier. In this example, thetable would be the successor object to the tree but it would berepresented as a different object in a different permanent class.

An example of a non-permanent class would be [lawyer]. A particularlawyer can only be an instance of this class for part of the time. Priorto qualifying (e.g. during his or her childhood) and perhaps afterleaving the profession they would not be a member of the class. Howeverhe or she is a member of the class [human being] for the entire timelineas [human being] is a permanent class.

It is also helpful to understand the difference between a permanentclass and the presence of a physical object in the universe for onlypart of the timeline. In the preferred embodiment an object isconsidered a member of a permanent class for the entire timeline evenfor parts of the timeline where that object isn't alive or doesn'texist. However, an object is considered a member of a non-permanentclass only for the time period when the relevant attributes/classmembership applied.

5.3.4.3 Principal Class

The principal class (PC) is a concept that exists in the preferredembodiment and various other embodiments.

A PC is a class which is considered the most useful in instantlyidentifying what sort of object something is. In general it should besufficiently specific a class as to give most of the common propertiesof an object, yet not so specific as to represent an obscure concept.Examples might include the class of human beings, nation states, trees,cities.

The PC is useful for quickly stating what an object is in a way that ahuman user will understand. In some embodiments it can be used by thesystem for identification purposes too. For example, if several objectshave the same name the system may use the principal class in combinationwith the name to uniquely identify the object to the user.

In some embodiments all objects must have a PC or having one is stronglyencouraged. A class cannot be a PC for some objects and not others whichare members of it (i.e. it is a property of the class). For this reasonwhen an object is added to the knowledge base and an assertion is madeabout a class of which the object is a member, there must normally be aPC on the way up the tree (if the asserted class itself is notprincipal). The PC of the object is the lowest (most specific) principalclass of which the object is a member.

One method for finding the principal class of an object is first toidentify the classes of which the object is a member, i.e. a query isdone looking for objects to which the entity has the relation [is aninstance of]. The resulting class objects are then ordered using the [isa subclass of] relation and the most specific class labelled as aprincipal class is then considered the PC for the object.

Principal classes are organised so that they are distinct from any otherprincipal class at the same level in the ontology so there are nocomplications with overlapping (non-distinct) classes which wouldprevent identifying a single principal class for the object.

A similar check is done while adding a new object when prompting theuser entity for a class of which the object is a member. After promptingthe user entity for a class, both this class and the classes to whichthis class is on the right in the relation [is a subclass of] areretrieved from the knowledge base and again they are ordered. The mostspecific class labelled as a PC is taken as the object's PC. If one isnot found using this method the user entity is prompted for a morespecific class, e.g. the string “policeman” will find [human being] asthe principal class (the class of policemen is a subclass of the PC[human being]) but “living thing” will result in the user being promptedto be more specific.

5.3.4.4 Relationship Between Attributes and Classes

In the preferred embodiment classes can also be represented in terms ofattributes. For example, being a member of the class [human being] canalso be thought of as having the attribute [human]. In this example asingle attribute is equivalent to class membership. For some classesmore than one attribute may be equivalent. For others a Boolean equationof attributes may define class membership.

5.3.5 Data/Document Objects

Some useful objects are essentially data. Examples include pictures,HTML pages, longer blocks of text or other documents. It is impracticalto name these objects using the data itself so in the preferredembodiment we give the data an object name. Having done that theinternet already has a well-established way of retrieving the contentsof such a file: URLs. URLs are named within the invention by using aparametered class [url] with a single-string object parameter, e.g.[url: [“http://wwwsemscript.com/”]]. The relation [is a url of] relatesthe object name for a document to a URL which contains the document'sdata.

5.3.6 Facts

Core to the preferred embodiment knowledge representation method is thefour object fact. The basic syntax is:

[name of fact]: [object 1][object 2][object 3]

i.e. four objects listed in order on one line, with a colon after thefirst one. A great deal can be achieved without the [name of fact]object and an alternative embodiment could omit this extra identifiercompletely, make it optional or infer it (say) from an identifier in thedatabase. However, in the preferred embodiment it is compulsory.

Object 1 and Object 3 can be of any type. Object 2 has to be a relation.This fact itself is an object with the name [name of fact]. Whenasserting knowledge all four objects have to be names.

In the preferred embodiment, the names of facts are of the form

[fact.<unique string>@network.machine.name]

The network machine name (e.g. an internet host name) “owns” the factand is responsible for its truthfulness and maintaining it. This hasutility for embodiments using a distributed model of multiple servers,each holding different facts, other machines would refer to this machineto get information about this fact if necessary. An alternativeembodiment would associate the machine with the fact but include thename of the machine separately from the fact name.

Here are some simple examples of facts:

[fact.2143@semscript.com]: [alejandro toledo] [is the president of][peru] [fact.1147758@semscript.com]: [male] [applies to] [abrahamlincoln] [fact.10@semscript.com]: [paris] [is the capital of] [france]

Note how facts are essentially strings of text and are thus easilystored on a computer and communicated over a network between computers.

The other advantage of the fact concept is its lack of complexity. Asequence of four objects with an extremely straightforward syntax can beregarded as a permanent atom of knowledge. An unordered collection ofsuch atoms can communicate and permanently store real knowledge withoutany of the problems of natural language.

Yet another advantage of the representation is that facts such as theabove can easily be stored in a standard relational database consistingof four columns with each field being text. Use of indexes means thatcombinations of known and unknown objects can rapidly be looked up. Afurther advantage is that as each atom of knowledge has a name, it isvery easy to represent facts about facts. This is typically how time isrepresented (see section 5.3.7 below) but could also include knowledgeabout when the fact was added to the knowledge base, what person orentity added it or any of a large number of other possible assertions.The naming also gives a source that “owns” the fact enabling all sortsof possibilities relating to maintaining and verifying the fact over anetwork.

Note also that by placing the relation object in the middle of the twoother objects, the facts “read” like natural language making it veryeasily understood by humans.

Knowledge stored in this manner can also be used to supplement knowledgestored using other strategies.

5.3.7 Time

Natural language generally asserts or implies tense relative to thepresent. However, in the preferred embodiment static knowledge can bestored long term and we express time in absolute terms, i.e. we assertthat things are true for periods or moments of time expressed as adate/time-of-day and not relative to the moment when they are expressed.i.e. Temporal data is associated with facts which in the preferredembodiment assert when the facts are true. Alternative methods arepossible but doing this avoids the complexity of having to adjust themeaning of facts from moment to moment as time goes by.

In the preferred embodiment most simple facts have no information abouttime implied or represented with it. For example:

[fact.1132040@semscript.com]:[london][is the capital of][united kingdom]

simply asserts that the relation [is the capital of] was/is/will be truefor at least one moment in the time line. It may be true for all of it,it may have been true only for an instant in 1658.

To get around this problem each fact expressing a relationship that canchange can be accompanied by one or more temporal partners. Temporalpartners are facts that reference other facts and make assertions aboutwhen another fact is valid, i.e. we represent the temporal data aboutwhen a fact is true with one or more further facts.

e.g.

[fact.2143@semscript.com]: [alejandro toledo] [is the president of][peru] [fact.2144@semscript.com]: [fact.2143@semscript.com] [applies fortimeperiod] [timeperiod: [timepoint: [“2001/7/28”]]; [iafter]][fact.2144@semscript.com] makes an assertion about[fact.2143@semscript.com] namely that Alejandro Toledo has been thepresident of Peru from the 28th of July 2001 to the indefinite future.Note that these two facts by themselves say nothing about whether or nothe was president before that. (To do that requires a similar pair offacts using the negative version of the fact for the period before 28thof July 2001. See section 5.3.8)

Many facts are true for all time and it is often possible to infer thisfrom attributes of the relation in the fact, e.g. the relation [is thebiological father of] is a permanent relationship: it does not makesense for someone to cease to be someone's genetic father or to begin tobe one when one was previously not. This practice of making an assertionwithout temporal partners is usually reserved for facts that absolutelycannot change from moment to moment—if they were true once they arealways true. Generators (see section 5.4) can then infer the temporalpartners (if needed) asserting that they are true for the whole timeline.

In addition to permanent relationships there are also permanentattributes. For example, the attributes [young] and [asleep] areexamples of transient attributes. [blood group o] and [sagittarian] areexamples of permanent attributes. Attributes which apply to arelationship and which are a consequence of their semantics, such as[symmetric], are permanent.

A third way of dealing with time is to not bother and for the knowledgebase only to attempt to represent information about the present (and notstore information about things that were true in the past and not truenow). Such an approach would require updates to the knowledge base whenthings change, replacing, removing or suppressing the facts that are nolonger true. “True-now methodology” is the term used herein for thisapproach. Such a fact simply asserts something about the moment in timewhen the access of the knowledge base was made and it is not necessarilytrue at any other time. In other words the ability to assert knowledgeabout the past is sacrificed in exchange for benefits of convenience andefficiency. In the preferred embodiment the attribute [relation istrue-now] labels relations where this methodology is used. Typicallywhen such relations appear in queries (see section 5.3.13), the querieshave no corresponding temporal partner so no inference is needed. Use ofsuch relations in a query is equivalent to asking about whether therelationship is true now without this having to be explicitly stated.

In the preferred embodiment, this method is used for facts used fortranslating to and from natural language. The reason being partly thattheir use is in translating questions and statements that happen in thepresent and thus old versions of these facts are not very useful, partlybecause they would almost never be used and partly because they changevery infrequently. Temporal partners could be included but it wouldneedlessly complicate the translation process. Another common situationwhere this method is (has to be) used is when querying the system forthe current time. A temporal partner for such a fact would be pointless.(An alternative approach for translation knowledge is to make suchrelations permanent. Although not strictly true, in practice words don'tchange their meaning very frequently and this approach is practical in asimilar way.)

A third situation where true now methodology is used is when thesemantics of the fact are based partly or entirely on what is in theknowledge base. For example, the relation [is a direct subclassof](whether one class is immediately below another in the ontology) hasthe attribute [relation is true-now] as its meaning is affected bywhether an intervening class is present in the knowledge base. Thisrelation could exist between two classes and then cease to exist whensomeone inserted a new class between them. As it is impractical to keeptrack of the dates and times of such events it is simpler just to usetrue now methodology in this case.

Another situation is temporal partners asserting a time periodterminating with the [after] object. As this can be closed at any timesuch an assertion uses true now methodology. For example a temporalpartner using the object [timeperiod: [timepoint: [“1987”]]; [iafter]]asserts the time period from 1987 until the indefinite future. At somepoint after the fact was added to the knowledge base, the fact may ceaseto be true. At this point the fact ceases to be true and a new assertionneeds to be made with the closing time period being an absolute timepoint. (Other embodiments could simply update the fact rather thanasserting a new fact and labelling the old one as false.)

The [timeperiod] class is a class of parametered objects where the twodescriptive objects are the point in time when the period of timecommenced and the point in time when it finished. However, to coverinfinite or indefinite periods of time there are three special timepoint objects in the preferred embodiment. The first is [iafter] whichindicates an unknown point in the future. It is used for things that aretrue at the time they were asserted but which are not guaranteed toremain true. The second and third are [time zero] and [forever] whichindicate respectively a point in time infinitely long ago and a point intime in the infinite future. They are used to indicate infinite periodsof time, for example the object [timeperiod: [time zero]; [forever]]indicates the entire time line and would be used, for example, in atemporal partner for facts that are true by definition.

In addition to [time zero] the preferred embodiment has a specialtimepoint called [earliest meaningful point]. This is useful forsituation where the user may not know or care about the timepoint whenthe relationship started but knows it was always true for as long as thefact could have been meaningful. In these situations [time zero] may beinaccurate and the alternative would be to just assert a recent timepoint when the user was sure the relation was true without saying itwasn't true before. An example would be asserting that the English cityof Cambridge is geographically located within the English county ofCambridgeshire. Neither Cambridge nor Cambridgeshire have existed forall time but for as long as they both existed one has been locatedwithin the other. [earliest meaningful point] thus saves the user frominvestigating what this earliest meaningful date might be.

In summary, within the preferred embodiment, facts are categorised aseither permanent, true-now or transient. Permanent facts have one of theforms:

<anything> [is an instance of] <permanent class> <anything> <permanentrelation> <anything> <permanent attribute> [applies to] <anything><anything> [applies for timeperiod] [timeperiod: <fixed start>; <fixedend>]

True-now facts have one of the forms:

<anything> <true-now relation> <anything> <true now attribute> [appliesto] <anything> <anything> [applies for timeperiod] [timeperiod:<anything>; [iafter]]

Anything not matching one of the above patterns is considered transient.

5.3.7.1 Other Temporal Classes

The permanent, transient and true now temporal classes are believedsufficient for a useful and practical model of the real world. However,other temporal classes which may be used in various embodiments aredynamic which means that the knowledge changes very quickly. Examples,would include financial data like stock prices. By identifying facts asdynamic, users can be prevented from entering them in the staticknowledge base.

Another class is unrevertable which is for facts which can become truebut once they do become true they continue to be true forever. Oneimplementation of this is for a generator to generate an associatedtimeperiod which ends with the [forever] timepoint instead of the[iafter] timepoint. An example, would be the [dead] attribute which doesnot apply to living things when they are alive but then applies forever.

For an embodiment to make use of these additional temporal classes onemethod is to create attributes for the attribute or relation in asimilar way to how permanence and true-now is implemented in above. e.g.with the attributes [relation is dynamic] or [attribute is unrevertable]

5.3.8 Negative Facts

In the preferred embodiment, it is also possible to assert that arelationship is not true. In the preferred embodiment this is done byputting the tilde (“˜”) character before the relation object, e.g.

[fact.3@semscript.com]: [london] ~[is the capital of] [england][fact.4@semscript.com]: [fact.3@semscript.com] [applies for timeperiod][timeperiod: [time zero]; [timepoint: “1066”]]

These two facts together assert that London was not the capital ofEngland before some point in 1066.

When storing facts in a relational database, representing negative factscan be achieved by the addition of a Boolean field to the table storingthe facts—when set true the fact is negative.

5.3.9 The Golden Rule

The Golden Rule is that a relationship cannot both exist and not existbetween the same pair of objects at the same moment in time.Contradictions or inconsistencies in knowledge represented by facts areproduced by finding or logically generating breaches of this rule.

Note that the representation of a timepoint is imprecise no matter howaccurately it is specified. In order to create a contradiction we haveto show that a relationship between the same pair of objects bothexisted and did not exist for two overlapping periods of time implied bythe accuracy of the timepoint. For example the British queen Victoriawas both alive and dead (not alive) in 1901: she was alive in the partof 1901 before her death and dead in the rest of it. If someoneremarries an hour after their divorce goes through they are married totwo different people on the same day but without being marriedbigamously. If, however, you can show that someone was alive for onetimeperiod and dead for another and show that the two time periodsoverlap, only then have you found a contradiction.

In the preferred embodiment, this golden rule is used to answer “no” toyes/no queries. See section 5.5 for details.

5.3.10 Categories of Knowledge

Various embodiments of the system classify knowledge into certaincategories in order to determine appropriate policies and actions forfacts within these categories.

Various embodiments can analyse a fact to determine (in at least somecases) which of these categories it falls into and act accordingly (e.g.when assessing the reliability of a fact or the penalties for it laterbeing contradicted). For example, the [uniquely translates as] relationis always associated with true by choice facts.

5.3.10.1 True by Definition

Some facts stem logically from the semantics of the entity andrelationships involved and don't need any external evidence to supportthem. For example, saying that the class of children is a subclass ofthe class of human beings stems from the definition of the two classesand the semantics of the [is a subclass of] relation. Similarly, sayingthat [symmetric][applies to][is married to] is a true by definition fact(the fact that A is married to B implies that B is married to A is aconsequence of the meaning of the relation).

5.3.10.2 True by Declaration

These are facts that are defined by the fact that a source of authoritysays so (and being generally accepted). Example includes the capitalcity of a country.

5.3.10.3 True by Choice

These are facts within the knowledge base where different people couldplausibly produce different answers and yet all be right. However, onlyone version can be used. A good example is the unique recognition andcommon translation strings for an object. This is similar to true bydeclaration except that the source is the user asserting the knowledgerather than any kind of accepted external authority.

5.3.10.4 True from Evidence

These are facts that don't stem from definition or declaration but comefrom observations made by people. The geographical location of an objectand whether a human being is male or female are examples of this classof knowledge.

5.3.10.5 Unprovable Knowledge

Some knowledge is widely believed and may be asserted by users of thesystem but there is no evidence or way of proving the fact either way.Some facts associated with a particular religion fall into thiscategory. Legend, mythology and historical facts with very poor evidenceto support them may also produce facts that have similar issues. Thiscan be dealt with by use of a context (see section 5.3.10.7 below fordiscussion of fictional knowledge). Users can choose to add contextsincluding their belief system into knowledge considered when theirqueries are answered. For example, members of a particular religioncould have the id for that context in the list of contexts to be usedwhen they are interacting with the system. Once they had authenticatedthemselves (see section 5.9) this list would be considered by theengine. The default list of contexts would just include one, though—the“base” context. Having a context distinct from “base” does not implythat the contents are not part of reality (though this is true withfictional contexts). It implies that they are unprovable using thestandards of evidence embodied in the policies of the system andadditionally disputed by significant numbers of people. It thus makessense to store these facts separately from facts that are likely to beuniversally accepted.

5.3.10.6 False Yet Believed Facts

Another category is things that are widely believed, yet also untrueaccording to the strong preponderance of evidence. Some urban mythswould fall into this category. In the preferred embodiment these areessentially untrue facts that would be dealt with like other factualknowledge with strict policies for what is needed before they can beasserted and removed using the same methods by which other knowledge isremoved. In other embodiments special policies may be needed forknowledge that appeared in this class.

5.3.10.7 Fictional Knowledge

These are “facts” that are true only in the context of a fictional work.An example would be an assertion of the address or birth date ofSherlock Holmes.

5.3.11 Contexts

Various embodiments can store and process fictional knowledge in theknowledge base by clearly labelling all “true by declaration” and “truefrom evidence” facts as belonging to a specific context (e.g. afictional movie or novel). This way inappropriate facts can be ignoredby the query processing system unless the query is specifically aboutthe context requested. When a specific context is part of the query, all“true by declaration” and “true from evidence” facts not belonging tothat context can be ignored and correct answers returned. “True bydefinition” knowledge can be used across contexts, even fictional ones.This method can also be used to extend the knowledge base to includefacts belonging to contexts which are not strictly fiction but wouldotherwise fail to be considered as fact.

In the preferred embodiment the unique recognition string (see section5.3.2.1) of a fictional object must make this clear to avoid anyconfusion. Thus the unique recognition string for [sherlock holmes]might be [“My fictional detective Sherlock Holmes”].

Contexts can also sometimes be inferred directly from a reference in aquery to an object or relationship that only belongs to one particularcontext. For example, the question “What is the address of SherlockHolmes?” would infer the context from the reference to the fictionalcharacter. “True from evidence” facts include the assertion of the classmembership of an entity (e.g. of its principal class) so the fact[sherlock holmes][is a member of][human being] would be associated witha fictional context and not the base context.

Some embodiments also use contexts to store conflicting “true bydeclaration” and “true by choice” facts. For example, when two differentauthorities disagree. Users can then resolve these conflicts byselecting contexts which they wish to be used when queries are answered.These selections can be permanently associated with a user and useduntil the user changes them. Knowledge associated with a particularreligion can be modelled this way by associating it with a contextpertaining to that religion.

5.3.12 Summary of Knowledge Representation

In the preferred embodiment of the invention the universe is modelled asa huge array of objects and relationships between pairs of objects. Asthe time clock ticks, named relationships between pairs of objectsspring in and out of existence.

Some of those relationships are known to exist at a particulartimepoint, some of those relationships are known not to exist at aparticular timepoint (negative facts) and with others the embodimentdoes not know. The more knowledge that is in the static knowledge baseand the more knowledge that can be generated, the more complete theknowledge is.

All possible objects “exist” in the knowledge base for all time.Existence of physical objects (and other objects such as legal entitieswhich only have life for a given period of time) is expressed by thepresence of a relationship with an attribute for the time they arearound in the real world, e.g.

[fact.1356@semscript.com]: [alive] [applies to] [queen victoria][fact.1357@semscript.com]: [fact.1356@semscript.com] [applies fortimeperiod] [timeperiod: [timepoint: [“1810”]]; [timepoint: [“1901”]]][fact.1358@semscript.com]: [alive] ~[applies to] [queen victoria][fact.1359@semscript.com]: [fact.1358@semscript.com] [applies fortimeperiod] [timeperiod: [timepoint: [“1901”]]; [forever]]

It should be noted that relationships can exist between physical objectseven when one is not present in the physical world any more, e.g. amodern-day book can be about Isaac Newton even though he died hundredsof years ago. The book and the person are both physical objects with arelationship between them existing in the present yet there is no sharedmoment in time when they both existed.

5.3.13 Queries

Queries are a machine-readable representation of a question. i.e. datawhich communicates to an embodiment what knowledge is desired. A numberof representations are possible and the representation will often be atleast partly determined by the chosen knowledge representation method.In the preferred embodiment, queries look very much like a series offacts but the purpose is to see whether they can be justified fromknowledge found in, or inferred from, the knowledge base rather than toassert information. Variables can also replace objects in the facts(including objects within parametered objects). For example:

query f: [abraham lincoln] [is married to] [mary todd lincoln] f[appliesat timepoint] [timepoint: [“1859/5/3”]]asks the question “Was Abraham Lincoln married to Mary Todd Lincoln onthe 3rd of May 1859?”.

Notice how the name of the first fact is given a variable f rather thana name. In processing the query the engine will solve for f with thefact name (if any) that asserts that relationship between AbrahamLincoln and Mary Todd Lincoln, and then try to satisfy the second lineusing it. Provided both lines can be satisfied with at least one valueof f the query will answer “Yes”.

In queries the name of a fact can be dropped when the name isunimportant, i.e. when it is not needed for a later part of the queryand when it is not wanted by the user. This is effectively the same asexpressing the fact name as a variable and then not referencing thevariable again. Such lines thus have only three objects listed.

Variables can also be used in place of other objects in the facts. Forexample:

query a f: a [is married to] [abraham lincoln] f[applies at timepoint][timepoint: [“1859/5/3”]]asks the question “Who was married to Abraham Lincoln on the 3rd of May1859?”.

If the query is requesting objects as the answer, one or more variablesthat represent the desired objects follow the “query” statement. If thequery is just trying to determine the truth of the query it is termedherein as a truth query and it has no variables after the “query”statement. Queries whose purpose is to produce one or more named objectsas answers are termed object queries. The above query with the a missingfrom the query statement would ask the question “Was anyone married toAbraham Lincoln on the 3rd of May 1859?”.

Note that the query representation is extremely elementary in form andyet also extremely expressive in what questions can be represented. Thissimplicity in form has many advantages for automatic processing and theefficacy of additional techniques. Embodiments with more complicated oradditional syntax in the query—e.g. with constructs taken from logic orprogramming languages—would fail to have these advantages. Note alsothat this simple representation means that the semantics of the query isunrelated to the order of the lines. Each line places a constraint onthe value or values of each variable within the line. The collection ofconstraints define the information being sought and the query headerspecifies what variable values are the results of the query. Althoughthe semantics of the query is unaltered by the line order, some linesmay need to be processed prior to other lines in order to obtain resultsfrom the knowledge base. The query processing engine is thus free toreorder or chose to process lines in a different order should the querybe presented in an order which cannot be processed.

A more complicated query is the following:

query a a [is an instance of] [nation state] t: a [is geographicallylocated within] [the continent of Europe] t [applies at timepoint][timepoint: [“1999”]] t1: f [is the capital of] a t1 [applies attimepoint] [timepoint: [“1999”]] f [commonly translates as] d c [is thefirst letter of] d c [equals] [“p”]which translates as “Which continental European countries have capitalcities whose names start with a ‘p’ in 1999?”.

The first line will generate a list of several hundred possible valuesfor a (current and former countries) which will be whittled down by thetests in the next few lines (for location within Europe, etc.). Thecapital cities are looked up, translated into strings which are theirusual English names and the first letter is checked to be a “p”. Anyvalues of a remaining after the last line is checked are returned by thequery.

Essentially lines in the query can be regarded as filters if theyreference variables that have been mentioned in earlier lines. Suchlines reduce the possible values for that variable by doing tests on it,substituting in all previously found values one by one and seeing if theresulting fact can be found (directly or after inference) in theknowledge base. If the line uses a variable for the first time it can beregarded as something that generates values—finding all possible valuesfor the variable that are passed downwards. If any values (orcombinations of values) survive the generating lines and filters to theend of the query they result in a “Yes” answer for a truth query, or alist of objects for object queries.

5.3.13.1 Parameters

The preferred embodiment also contains certain parameters that can beadded to lines in a query for efficiency and other reasons. Theseinclude:

/s

means that the current line should only be processed using staticknowledge. There is no need to use knowledge generation to find this out(see section 5.4). A typical situation for this is to see whether acommon attribute applies. If the attribute is a fundamental propertythat can be assumed to be always stored statically if it applies, thenthere is no point in doing anything more complicated to find it, e.g. aline in a query might be:[symmetric][applies to]r/swhere r is a relation.

If a relation is used, its core properties are always stored so we canassume that [symmetric] either applies or not from a static search anddo not need to waste time trying to use inference to see if it holds.

In the preferred embodiment this parameter also enables the query to“see” superfluous facts which have been labelled as invisible.

/1

means that only one answer need be found for this line (i.e. onesubstitution for the variables). Two possibilities for using this areeither that the semantics of what is being asked implies there is onlyone answer, or that only one answer is needed. This increases theefficiency of the engine as the search can stop after the first objectis found./emeans that the query will only be matched on static true facts which arenot superfluous. One use of this parameter is to easily implement agenerator for the [is a direct subclass of] relation by checking for [isa subclass] between the classes with /e. As [is a subclass of] istransitive, a static assertion of [is a subclass of] between two classesthat have an intervening class will be labelled as superfluous by thesystem assessment component (see section 5.11 for details of superfluousfacts). As ignoring these is part of the semantic definition of [is adirect subclass of] this is the desired behaviour.5.4 Knowledge Generation

Far more facts exist than can be stored statically. Various embodimentsof the present invention can generate facts not asserted directly in thestatic knowledge base usually (but not exclusively) by referencing andinferring these new facts from facts in the static knowledge base (andpossibly other generated facts).

One method of doing this is to hard code the generation rules usingprogram code. The preferred embodiment takes a more flexible scheme byusing generators.

As used herein, a “generator” is a stored entity used by the knowledgegeneration system to generate facts not present in the static knowledgebase. In the preferred embodiment, a generator has one or more targetlines which specify a pattern for the facts that can be generated bythis generator (these are termed “target lines” herein) in combinationwith mechanisms for generating facts that match this pattern.

In a “dumb generator” such a mechanism may simply be a query. The querygives values to the unknowns in the target line or lines and the resultsof the query are substituted into the target line (or lines) to generatethe facts, if the query is successful. In a “smart generator” there issome program code (termed a “tool” herein) optionally in combinationwith a query which is used to generate the facts.

Smart generators are like dumb generators but where the footer cannot begenerated simply by rewriting the results of a query: some computer codeneeds to be executed (run directly, or interpreted) to produce theresults.

To put it another way, the query format of the preferred embodiment,although very expressive, is not Turing powerful. This has manyadvantages in terms of efficient processing of the query but means thatsome inference steps cannot be achieved without additional processing.By adding a Turing powerful step to the header query, as described here,the full universe of possible inference steps can be achieved.

Many embodiments are possible in representing generators but in thepreferred embodiment generators are specified in detail, in three parts:

a title line identifying it as a generator and listing variables;

(if present) a header query that must be run to see if the generatorapplies (and possibly to generate values in a dumb generator—the list ofvariables after the generator line);

and a footer which is a number of facts which can potentially begenerated by the generator.

In the preferred embodiment lines starting with “!” are used forcomments and are ignored by the engine.

A simple example of a dumb generator is the following:

generator a%,b%,tp f: a% [is married to] b% f [applies for timeperiod]tp => t: b% [is married to] a% * t [applies for timeperiod] tp

This asserts that if person a is married to person b for a given timeperiod then person b is also married to person a for that same timeperiod. This might be obvious for marriage but it would not be true forexample with the relation [is a parent of].

This example is illustrative. In the preferred embodiment the aboveexample is carried out by looking for the attribute [symmetric] to therelation and having a slightly more complicated generator that onlyswitches the two objects around if the attribute applies. This saveshaving to have a similar generator for every relation with thisproperty.

i.e.

generator a%,b%,tr [symmetric] [applies to] r$ /s f: a% r$ b% f [appliesfor timeperiod] tr => g: b% r$ a% * g [applies for timeperiod] tr(plus a similar one for the negative relation)

Dumb generators express inferences about how, for example, the existenceof a relationship implies the existence of other relationships or howthe existence of an attribute can be used to infer other facts.

For efficiency reasons it is clearly not practical to run everygenerator that exists at every opportunity so when answering a line of aquery, the query answering system first checks information storedstatically, and then goes on to look at generators later by matching theline of the query it is currently on with lines in the footer of thegenerator (i.e. it works backwards). Only the lines marked with anasterisk can be matched. If the line matches, the top of the generatoris run as a query (perhaps with values substituted for variables) to seewhether the bottom lines can be considered as facts. If they are, thefooter facts are generated and the generated facts are added to a cache.Any objects that match variables are included in the answering of thequery.

In the preferred embodiment, the character that ends a variable nameindicates rules on what can be matched with it. Sometimes, whencomparing the current line of a query with the asterisked footer line, avariable will match a variable, sometimes a named object will match avariable, and sometimes a variable will match a named object. Suchmatches can happen within parametered objects as well as at the toplevel.

The percent sign after the variables in the matched line says that thevariable can be either left as a variable (i.e. matched with a variablein the query line and filled by the query in the top half of thegenerator) or textually substituted for a name. If substituted, thevariable is removed from the query statement at the top, and the objectname is substituted into the header query wherever the footer variableappears.

For example, if matching [mary todd lincoln][is married to][abrahamlincoln], the above generator would be run and the top half would be aquery getting all the timeperiods on file for when they were married.Facts would then be generated asserting that they were married the otherway around for the same timeperiods.

A dollar sign following the variable says that the variable must bereplaced and textually substituted for a real object name from the queryline being looked at—matching with other variables is not permitted andthe generator will not be used if that is the kind of match found. Ifthe variable has no percent or dollar sign it must correspond to avariable in the query line. By ‘must’ we mean that we cannot use thegenerator if the correct match is not present.

The unique fact names for the results of a generator are createdautomatically by the inference engine and are assigned to variables ifthey are needed for temporal partners (as with the above example). Factsgenerated by generators are also inserted into a temporary cache by theengine so they can be quickly found for use in subsequent processing ofthe query. This cache is checked by the engine even before searchingstatically-stored local facts. The cache enables facts generated inearlier parts of the query to be accessed without running the generatora second time with the same objects. By keeping a record of whatgenerators with what parameters generated items in the cache, the enginecan avoid doing the same operation twice simply by using the cacheitems.

As an example of a smart generator, a very commonly executed generatoris the following:

generator tr a$ [applies for timeperiod] tr =>timeperiod_to_timepoint@local a$ [applies at timepoint] tp$ *which says that a relationship is true at a timepoint if the timepointlies within a timeperiod when the relationship is true. This generatoris vital as it simply is not practical to list, say, every instant whentwo people are married as there are an infinite number of instants inany time period. We instead statically store a period of time and if aquery asks whether they are married at a given instant the above smartgenerator is put into action. First, all the timeperiods are gatheredusing the query at the top and the results passed to thetimeperiod_to_timepoint tool (essentially an executable function) withthe timepoint and timeperiod in question passed as parameters. If thetool determines that the time point lies within the timeperiod, itgenerates the footer with an appropriate name for the newly-generatedfact, otherwise it does not. Note that it is not possible to do thisdirectly using a dumb generator as calculation is needed to determinewhether one point in time lies within a named time period.

Note also that the dollar character at the end of the tp$ variableimplies that it must be matched with a named object—a named timepoint.There are an infinite number of timepoints in any timeperiod so it isonly possible to check a given timepoint, not to generate all thepossible answers.

Another commonly-used smart generator is the following:

generator => now@local [current time]  [applies to] n *which works out which time point is the current time. This smartgenerator does not need a query at the top (the query is null and can bethought of as always returning “yes”). The reason is that we can alwaysbe sure that there is a timepoint which is the current time. Thegenerator just generates the footer with the system date and time as thetime point whenever it is called. Naturally the current time cannot belooked up statically.

Smart generators can also be used to retrieve highly dynamic knowledgefrom a conventional database. For example, a smart generator could bewritten to return the current share price of a particular company byquerying systems in the stock market. (This knowledge in turn may beused by another generator to calculate the company's marketcapitalization.) In this case, as with the example of the current time,the smart generator is retrieving knowledge from a third source ratherthan calculating from facts originating from the static knowledge base.

Far more sophisticated smart generators can be written and we canpotentially incorporate all the algorithmic achievements of the planetinto the knowledge base for use in a very wide variety of contexts.

The computer code (“tool”) that provides the intelligence to the smartgenerator is named in the preferred embodiment byname@machine.on.internet

The machine.on.internet is a named machine which owns the tool and wherethe code can possibly be executed remotely. The term “local” refers tothe code that can be found on the local machine and/or is part of thelocal knowledge processing engine.

A wide variety of ways of implementing the tools and execution of thetools should be obvious to anyone skilled in the art. These includehard-coding of some of the more common tools within the engine; remoteexecution of code through network protocols (passing the parameters overthe network and receiving the results); text scripting languages thatcan be downloaded from a remote machine and executed locally; otherkinds of downloadable languages including those that are then executeddirectly on the local machine, etc.

In the preferred embodiment, the generator description is stored in arelational database which is accessed by the query answering system.

In the case of smart generators, the name of the tool identifies thecomputer code to run. Many tools are hard-coded within the system andnot accessible externally. However, the preferred embodiment also allowsfor users to add generators including smart generator tools using aninterpreted language and an approval step. This is described in moredetail in section 5.10.14.

Most generators only have a single target line. One exception is thefollowing:

generator => age@local f: a [is the age of] b$ * f [applies attimepoint] tp$ *which has two target lines to match as the age of something varies fromtimepoint to timepoint so the timepoint object needs to be known aswell.

An alternative embodiment could still implement this on a single linewith a relation that included the timepoint in the definition of therelation. e.g.

generator => age@local a [is the age of]  [group: b$; tp$]

This involves defining the right class of the relation as a combinationof the object and the timepoint of interest. In this embodiment, [is theage of] is a permanent relation, in the one above it is transient.

5.5 Query Answering

The way queries are answered is determined in part by the knowledgerepresentation and query representation method chosen.

Even within the preferred embodiment query representation and knowledgerepresentation method, a number of choices are available. However, thefollowing gives a detailed account of how queries are responded to inthe preferred embodiment:

5.5.1 Query Modes

In the preferred embodiment, queries can be run in a number of modes.Establish mode simply checks whether values can be found in theknowledge base that confirm the facts: “no” and “unknown” are thus thesame result for truth queries.

Full mode attempts to distinguish between “no” and “unknown” for truthqueries by seeing whether it can establish a breach of the Golden Rulefor any part of the query using facts in the knowledge base, by assumingthat the lines in the query are true. This test is done if it fails toanswer the query with its first attempt. If it can establish acontradiction the answer to the query is “no” as we can assume that theGolden Rule is true. If not, the answer is “unknown”.

This is done in the above query-answering algorithm by adding a testafter the first line of a query has failed to produce an answer. Thetest is designed to see whether the failure may simply be due to theknowledge base not containing the answer (an “I do not know” answer tothe Yes/No question) or because it is wrong and contradicted by otherknowledge in the knowledge base (a “no” answer to the Yes/No question).

The test is illustrated in FIG. 4. Step 402 involves searching for atemporal partner for the first line of the query. If there is one, step404 is performed: creating a reverse query by making the relationnegative (or positive if it is negative), and switching the semantics ofthe temporal partner between the concept of “within” and “for all of”for the corresponding timeperiod (or, in the case of a time point, thetime period implied by the accuracy of the time point). So, the [appliesat timepoint] relation is replaced by [applies for all of timepoint]relation and the [applies for timeperiod] relation is replaced by[applies for some of timeperiod] and vice versa.

In the case where there is no temporal partner (implying either apermanently true relation or one that is true at the present), step 406is performed: the reverse query created is simply the query line with apositive relation made negative or a negative relation made positive.

The reverse query created in step 404 or 406 is then run, and the resultexamined (step 408). A “yes” answer to the reverse query means that theroutine can answer the original query with a “no” (step 410). If theanswer to the reverse query is “no”, then the answer to the originalquery remains unknown (step 412).

For example, although it might be possible for both the facts “John ismarried to Sarah in 1999” and “John is not married to Sarah in 1999” tobe true (if they divorced in that same year) it would not be possiblefor both to be true if the second statement was instead “John is notmarried to Sarah for all of 1999” and in this case one statement beingtrue implies that the other is false.

The issue of completeness of results for object queries (i.e. have allobjects been returned?) is dealt with in section 5.5.2 below (it is alsotested for in full mode). To do this requires information in theknowledge base about the number of objects that have a particularrelationship which can then be matched with the number of objectsactually found.

5.5.2 Completeness

Completeness is the issue of knowing whether the answers that have beengiven by an object query are a complete list of all possible answers:there may or may not be answers which are not in the knowledge base. Itis checked for when a query is run in full mode.

The way it is done is by storing data about how many objects exist for atemplate line in a query. If that number of objects is found and allsubsequent filtering of them by successive lines in the query produces adefinitive yes or no result we can be sure that the objects that emergeat the end of the query are a complete list.

In the preferred embodiment, we store the data about number by the useof queryline objects.

Queryline objects are parametered objects that represent a possible linein a query (excluding the fact name). Each queryline object, therefore,has exactly three parameters. These parameters are either the specialobject [queryline unknown] which represents a variable or they are thenames of specific objects. For example, the possible line of a query:

n[is a child of][president james monroe]

and all similar lines with another variable are represented by thesingle queryline object:

[queryline: [queryline unknown];[is a child of];[president jamesmonroe]]

To say that President James Monroe has (has ever had) three children wethen include the following fact in our knowledge base:

[fact.000269@semscript.com]: [queryline: [queryline unknown]; [is achild of]; [president james monroe]] [has order] [integer: [“3”]]

When the engine comes across an object-generating line of a query (i.e.a line containing a variable that hasn't been used before), if it isasked to give completeness information to the query (i.e. if it is runin full mode), it does so by the process illustrated in FIG. 5. Itstarts by setting a flag indicating the completeness of the results tocomplete (step 502). This completeness flag can have three valuesmeaning that the results are complete, incomplete or the completenessstatus is unknown.

Next, it looks up whether any information is available on the number ofobjects known to exist for the query (step 504). In the preferredembodiment, it does this by converting the query to a queryline objectand running a second query to see whether there is a [has order] fact inthe knowledge base. If there is no information on the number of objects,the completeness flag is set to unknown (step 506), and that line of thequery is run (step 508); the flag will then stay unknown for theremainder of the query. If there is information on the number ofobjects, it compares the number of results found after executing theline (step 510) with the number of objects known to exist (step 512), asasserted by the queryline fact in the preferred embodiment. If theymatch, the completeness status is preserved as complete. If the numberof objects found is smaller than the number indicated, the flag is setto incomplete (step 514). (If larger, there is an inconsistency in theknowledge base, so the completeness is unknown, and the flag is setaccordingly—step 516.)

Step 518 checks whether there are further lines in the query. If thereare no further lines, the process simply returns the objects found, andthe status of the completeness flag. If there are further lines, then,for as long as the completeness flag remains complete, the engine doesextra work to determine whether the results it has found so far continueto be complete.

Subsequent lines in the query may filter the objects found (i.e. theline may include only a variable used to generate the objects on aprevious line so when reached it substitutes the previously foundobjects in and only ones which can be justified survive). Beforeproceeding to execute subsequent lines, the completeness status ischecked (step 520).

If the completeness status going into a filtering line is unknown, theremaining lines of the query are executed (step 522), but no furtherchecks on completeness will be undergone (the flag remains set tounknown).

If the status is incomplete, the completeness status changes to unknownafterwards no matter what the result (step 524): we do not know whetherthe missing objects would have passed through the filter or not withoutknowing what they are.

If the completeness flag is set to complete it then becomes important todo extra work if the object fails to pass through that line (step 526).If the answer can be shown as a “no” then the completeness status of thequery so far is unchanged. If, however, it is unknown, then thecompleteness flag has to be changed to unknown as well. The method usedto determine between “no” and “unknown” is exactly the same as the oneused to answer a truth query with “no” described above (and illustratedin FIG. 4): essentially the relation in the query line is made negativeand any temporal partner is added to cover all of the timeperiodspecified—if this new query is found to be true we can answer “no” tothe original mini-query and preserve the status so far as complete.

For an example of completeness information being used in an embodimentof the invention see FIG. 7. The answers in screen 702 are introducedwith the text “There are 3 answers to this question. Here is a completelist.” Without the completeness check it could only introduce the listwith “Here are the answers I found.”

5.5.3 Explanation of an Answer to a Query One of the desirable (butoptional) features of various embodiments (including the preferredembodiment) is the generation of a justification for its answer to aquery.

Such explanations are a helpful feature because they demonstrate wherethe answer “magically” produced came from, thus greatly improving theconfidence the user has in the result. Moreover, although the resultsmay have come from a computer, a human being ultimately has to use thatknowledge and take responsibility for its accuracy.

Another advantage in embodiments which include user assessment (seesection 5.10.17) is that the user has a chance to see where an incorrectanswer came from and do something about the incorrect fact or facts thatresulted in that incorrect response.

The preferred embodiment is operable to produce two types ofexplanation: a detailed explanation which is essentially a step-by-stepproof of the answer and a concise explanation designed to give the usera hint about where the result came from. Other embodiments may produceone or the other (or none).

FIG. 8 shows an example of both types of explanation in an embodiment.(This figure is described in more detail in section 5.5.6.)

5.5.3.1 Detailed Explanation

In the preferred embodiment, the detailed explanation is essentially allthe facts and generators that it used to find the answers returned,turned into a natural language proof.

The way that this is implemented in the preferred embodiment is by useof a data structure which is a linked list of items where each item caneither be a string containing a line of natural language (typicallydescribing an event in the processing of a query), or a fact. This datastructure can either hold the entire explanation, or the explanation forsome part of the answering of the query.

During the processing of a query, many smaller queries are executedbecause many of the lines in the query involve the use or possible useof generators and the header queries in the generators need to be run.Some of these generator queries succeed and some fail—when they succeed,the explanation for those queries producing the used fact forms part ofthe parent explanation. To complicate matters further, some of thesegenerator runs produce facts which go into the cache, and the fact isthen subsequently found in the cache rather than from running thegenerator again. It is helpful that when this happens, the originalexplanation for how the fact was generated before being placed in thecache forms part of the final explanation, and if the fact has alreadybeen justified, it is not explained twice in the final explanation.

A full explanation is stored with every fact placed in the cache. When asuccessful search of the cache is made and the fact pulled out, a checkis made to see whether this fact has been previously used (and thereforejustified) in the explanation of the lines of the current query done sofar. If so, we just add a line saying something like “We already knowthat”; if not, we insert the entire explanation for the cached fact intothe parent explanation.

This same process of adding the lines of an explanation into the parentexplanation happens when we successfully generate facts from agenerator. (The same query processing routine is called for the queryheader.) When it returns, we take that explanation for the set ofresults we are using and insert it into the explanation for thegenerated facts with text lines to help understand how the facts werecreated. For example, we can insert a line like “Therefore:” along with(optionally) the name of the generator between the explanation for theheader query and the footer lines of a generator. As generators are alsocalled during the processing of queries in the headers of othergenerators, lines from explanation can end up being generated severallevels of query-processing deep.

The final step is to translate our data structure into the naturallanguage explanation. Translation involves the following three steps:

(1) Eliminating certain repetitive sequences for clarity. For example,with the pattern:

I know from static knowledge that:

fact 1

I know from static knowledge that:

fact 2

The third line can be eliminated.

(2) Translating the fact ids into presentable names. In the preferredembodiment, the static fact ids are long and the generated ones areunique internal strings derived from the objects in the fact. Neither ofthese are very printable. To make this more palatable, these ids aretranslated into local fact names, “fact 1”, “fact 2” etc. and a look uptable maintained so these names can be propagated through theexplanation.(3) Translating the facts into natural language. This can be done withthe translation string for the elements, e.g. substituting each objectfor the string which has the relationship [commonly translates as] withit. Fact id translation comes from the table created in step 2.

The explanations described above are not shown by default in thepreferred embodiment and are only displayed when a user requests it.

5.5.3.2 Concise Explanation

Various embodiments including the preferred embodiment can also displaya concise explanation. In the preferred embodiment this is just thestatically stored facts that were referenced on the way to answering thequery using a method similar to that described above but with all theinference steps and inferred facts not shown. In most cases the humanuser can intuitively understand any inference that was done and anyincorrect knowledge used to answer the query is most likely to be instatic form. (If the generator is incorrect in some way this can be seenwith the detailed explanation which can be selected by the user if theycannot understand what has happened.)

Other embodiments may include purely calculated facts in the conciseexplanation. Purely calculated facts are generated facts which are notinferred from static facts. e.g. They are facts which the generator hassourced from somewhere external to the static knowledge base.

In embodiments which enable users to endorse and/or contradict factsand/or display the sources of the static fact (user assessment), thisabbreviated explanation enables links to be placed next to the staticfacts referenced thereby allowing the user rapid access to thisfunctionality. The concise explanation is also often short enough thatit can be displayed under the answer to the question without occupyingexcessive screen space.

When a detailed explanation is generated, generating a conciseexplanation can be achieved by scanning the lines of the detailedexplanation and extracting out the facts that came from the staticknowledge base (avoiding duplication).

Alternative embodiments can generate a concise explanation from scratchwithout the need to generate a detailed explanation. In theseembodiments the concise explanation is generated by keeping track of theessential facts which were referenced while the query was beingprocessed.

Various embodiments may refine the concise explanation to include onlyan essential subset of the static facts referenced when answering thequery in order to make the information presented to the user even moreconcise. Candidates for elimination are the more unintuitive facts suchas properties of relationships which users may know intuitively anyway,e.g. [symmetric][applies to][is married to]. In embodiments which canrate the veracity of facts, facts whose veracity are not in dispute andwhich have these characteristics are an especially high priority forelimination.

5.5.4 The process_query routine

The semantics and syntax of the preferred embodiment query are describedin section 5.3.13, generators are described in section 5.4 and theformat of knowledge in the static knowledge base is described in section5.3. A number of ways of implementing a query processing system havingthese features and semantics is possible.

However, the implementation of query answering used in the preferredembodiment is now described.

Facts come from three sources: (1) the static knowledge base, (2) theknowledge generation system and (3) a cache of all facts previouslydiscovered when processing this query (and in some embodiments possiblyearlier than processing this query if the cache is not flushed betweenqueries).

The routines that retrieve from these three sources are static_search,generator_search and cache_search.

The implementation of each of these in the preferred embodiment is nowdescribed.

5.5.4.1 Searching the static knowledge base (static_search)

In the preferred embodiment the static facts are stored in a table in astandard relational database (the ‘facts’ table).

The table has the objects in the fact stored in fields id, left_object,relation and right_object. Each combination of these is indexed forspeed.

Additionally the table has the following extra fields:

negative: a Boolean field which makes the relation negative(corresponding to the presence of the tilde ‘˜’ character when the factis written out).

true: whether the system believes the fact is true (set by userassessment and system assessment—see below).

visible: whether the fact is being used to answer queries. All untruefacts are invisible and some superfluous ones are also invisible incertain embodiments.

superfluous: whether the fact can be generated by the system anyway.

contradicted: whether the fact is in semantic conflict with otherbelieved-true facts

challengeable: Boolean: whether further user assessment is allowed forthis fact.

last_update: the date and time of the last system assessment of thisfact.

superfluous and contradicted are set by system assessment. The truefield is set by system assessment (sometimes using user assessmentdata). User assessment is described in section 5.10.17. Systemassessment is described in section 5.11.

The parameters passed to the static_search routine are:

The queryline currently being searched;

A pointer to a list of facts into which the routine will place thestatic facts that match the queryline (i.e. a place to put the returnedfacts);

A pointer to a list of explanations to explain each fact returned;

A pointer to the query that is being processed;

When the routine is called it builds a SQL SELECT statement to retrievethe static facts from the table that may match the queryline.

For objects in the queryline that are fully specified objects, this is amatter of adding an element to the WHERE part of the statement thatspecifies this. For example, “where left_object=‘abraham lincoln’”.

When an element of the queryline is a parametered object and one or moreof the parameters are variables, some embodiments that wish to (andwhere their version of SQL supports it) can build a wildcard componentof the WHERE clause corresponding to the known and unknown elements. Forexample, if the left object in the query was [timepoint a] thecorresponding part of the where clause would be “(left_object like‘timepoint:%’)”. Other embodiments could just leave partially specifiedobjects out and eliminate by matching the returned results against thequeryline after the SELECT statement has received the candidates.

The WHERE clause also contains some conditions associated with the extraparameters. For most requests this will be “true=1 and visible=1”.However, this will be adjusted by any parameter settings for thequeryline (see section 5.3.13.1). For example, in the preferredembodiment when the “/s” parameter is used the “visible=1” isn'tspecified as some invisible facts are true and superfluous but as theparameter will stop any fact generation from being done it makes senseto use the fact anyway.

Another situation is when the queryline specifies the fact id. In thissituation even the true field isn't constrained as the semantics of aqueryline specifying a fact id are to enquire about the facts elements,not to ask whether the relationship is true.

The WHERE clause also needs to specify the negative field according towhether the relation in the queryline is positive or negative.

Once the SQL query has been assembled it is executed to retrieve a listof static facts. Each of these facts is then tested against thequeryline if necessary to ensure it matches.

The facts that match are added to the fact list with a simpleexplanation added to the explanation list. The explanation consists oftwo lines: “I know from statically stored knowledge that” and the factitself.

The facts and explanations are also added to the cache (if not alreadypresent). If already present, the explanation is substituted for thestatic one if the explanation in the cache is longer.

5.5.4.2 Searching generated facts (generator_search)

The generator_search routine receives as parameters the queryline and apointer to a list of facts and explanations where the matching generatedfacts are to be placed. In combination with the generators themselvesand tool implementations it forms part of the knowledge generationsubsystem in the preferred embodiment.

If the queryline ends “/s” generator_search simply exits. If it ends“/1” it exits if or when there is one returned value.

The first thing it does is assemble a list of generators that arecapable of producing facts which match the queryline provided. It doesthis by matching the queryline against the target lines of thegenerators and selecting the generators that have one that matches.

In embodiments where generators can have more than one line to match,the routine may need to scan later lines in the query to match againstthe other target lines once the first line has been matched. In theseembodiments, a pointer to the query will need to be passed to enablethis scanning.

For each matching generator it then does the following:

If there is a header query it:

-   -   substitutes any values into the header query from the matches in        the target line (if necessary).    -   removes from the list of header query variables any variables in        the target line which have been matched with a known object.    -   calls process_query on this header collecting all the values        returned.        If a smart generator it then:    -   passes each set of values into the tool and collects the facts        it generates.        If a dumb generator it:    -   substitutes each set of values generated by the query and        matches from the queryline into the footer lines to generate        facts. (Any variables for the fact id in a generated fact are        given values as this is being done.)

For each of the facts generated by either method it:

-   -   creates an explanation. This explanation is the explanation for        the set of values used, generated by the processing of the        header query, plus an introduction line, plus the facts        generated using this set of values. For dumb generators and        smart generators with a header, the introduction line is        “Therefore:” and the name of the generator. For smart generators        without a header query it is “By calculation:” and the name of        the smart generator.    -   stores the fact and explanation in the cache if it isn't already        there. If it is already there, it substitutes the explanation if        the newly generated explanation is shorter than the one stored.    -   if the fact matches the queryline, it adds the generated fact        and explanation to the results list.        5.5.4.3 Searching the cache (cache_search)

The cache is where facts previously found using the other two sourcesare stored.

The cache contains the facts and the best (shortest) explanationassociated with each fact.

The routine receives a queryline and a pointer to fact list andexplanation list as parameters. The facts in the cache that match thequeryline are to be placed in the fact list and their correspondingexplanations in the explanation list. As with the other two routines,the correspondence between the explanation and fact is established bythe ordering. e.g. The 5th explanation in the list corresponds to the5th fact in the list.

It also receives a pointer to the query being processed as a parameter.This enables the routine to keep the detailed explanation a littleneater by avoiding explaining the same fact twice.

The process_query routine maintains a record of all the queries that arecurrently being recursively processed by maintaining a pointer in thequery object that points to its parent query. Child queries are querieswhich are being processed to provide answers for another query. That is,a child query is the query that is formed from the remaining lines of aquery when the first line is resolved (see below for how this is done)or a query in the header of a generator called when processing aqueryline for a parent query.

The first thing this routine does is look up the matching facts in thecache.

If it finds a match it then scans to see whether this fact has beenexplained before.

The query object holds a ‘pre-explanation’ which contains theexplanation for a set of values which is pending while the remainder ofthe lines using those values are evaluated. It also contains a standardexplanation which is the partial explanation so far for the query.

By scanning up the linked list of queries defined by the parent_querypointer and for each query scanning both the pre_explanation andstandard explanation for matches, the cache_search routine can determinewhether this fact has been explained previously.

If it has been explained previously it simply creates a two lineexplanation for the fact. “We already know that:” and the fact. If ithasn't been explained before, it copies the entire stored explanationfrom the cache into the returned results.

A fast cache lookup (avoiding an exhaustive search of the stored facts)can be achieved in various embodiments by hashing.

As the queryline contains both known and unknown objects, oneimplementation is to hash each fact several times to enable fast lookupeven with the unknowns.

For example, one simple implementation designed to rapidly locate factsin the cache could create three open (externally-chained) hash tablesfor left_object, relation and right_object pointing at all facts with anamed object in the hashed position. Possible cache matches for aqueryline could then be located by looking up cache facts that match theknown object(s)/positions(s) in the queryline. A full check needs to bedone on the candidates but the hash tables would mean the number ofcandidates checked was substantially smaller than an exhaustive scan ofthe cache.

A faster implementation is to additionally create a hash table for eachcombination of two known objects, e.g. facts matching a querylinecontaining a known left object and known relation could be rapidlylooked up if all facts were hashed on their objects in those positions.

5.5.4.4 Operation of process_query

The process_query routine receives the following parameters:

A pointer to the query to be processed.

A pointer to a list of strings used to return variable results.

A pointer to an explanation list used to return an explanation for eachset of results.

The strings returned are in the form:

<variable>=[object]

The number of sets of results can be determined by dividing the numberof strings in the string list by the number of header variables in thequery. (For truth queries no variable values are returned.)

The process_query routine also returns a status value indicating thestatus of the query when processing has finished. The possible returnvalues for truth queries are:

Yes: the truth query can be satisfied.

No: It can be proven that the truth query cannot be satisfied.

Unknown: It wasn't possible to establish an answer to the truth queryeither way.

No/Unknown: For non-full mode this just means that it wasn't possible tosatisfy the query and that no attempt was made to answer “no”.

For queries with header variables the possible return values are:

complete: the list of values returned is all that exist.

incomplete: the list of values returned is shorter than what exists.

completeness_unknown: these are the values the system is aware of.

Both types of query can also return

error—when something goes wrong.

FIG. 6 shows the process_query method of the preferred embodiment. Thisfigure assumes the query is being run in full mode and that explanationsare being generated. (If it isn't, the steps necessary for completeness,answering no and generating explanations can be skipped.)

First a check is done on the number of lines in the query (602).

If there are no lines in the query yes/complete is returned (604) andthe routine ends. Otherwise the top queryline is retrieved (606).

In order to avoid infinite loops a record of all querylines currentlybeing recursively processed is maintained, the “unresolved stack”. Thefirst thing that is done with the queryline is to check whether it isanywhere in this stack (608).

If it is, unknown/completeness unknown is returned (610) and the routineends. Otherwise the queryline is added to the unresolved stack. (612).

An order is next obtained (if present) for this queryline (614). This isdescribed in detail above.

Next a search of the cache is undertaken (616). The cache_search routineis described in detail above.

The “queryline cache” is a record of all querylines that have beensuccessfully processed. By keeping a record of all processed querylinesand storing every result matched to a queryline in a cache, the staticsearch and generator search routines can be skipped when the querylinehas been processed before, making the routine more efficient. (For thisreason both the queryline cache and the fact cache must be flushedsimultaneously or not at all.)

In step 618 the queryline cache is checked.

If the queryline has not been cached the static and generator searchesare undertaken (step 620) and the queryline added to the queryline cache(step 622). (Either or both of these search routines may be skipped ifthe queryline ends “/1” and a fact has already been found.)

Control then passes to step 624 which sees whether the querylinecontains any variables and whether any matching facts have been found.

If there are no variables and no results, we test for “no” as describedabove (step 626) and return no/complete if successful (step 628) orunknown/completeness unknown if not (step 610). In either case, thequeryline is removed from the unresolved stack before completion (step611)

If there are results or variables in the queryline, control goes to step630 where a check is made to see whether there are any facts found whichmatch the queryline.

If there are no matching facts the routine returns unknown/completenessunknown (step 610).

If there are matching facts control passes to step 632 where duplicatefacts are removed. If there are duplicate facts the one with theshortest associated explanation is the one kept.

Control then proceeds to step 634 where a provisional return result isset. If it is a truth query the provisional result is yes; if an objectquery and the order isn't known, the result is completeness unknown, ifan order query and the number of matching facts matches the order theresult is set to complete, otherwise the result is set to incomplete

What remains is to process the remaining lines of the query.

In the preferred embodiment, this is done by creating a child query foreach matching fact consisting of the remaining lines with the values ofthe variables implied by the matching fact substituted into theremaining lines, executing each query and carefully preserving theresults and explanations before and after (step 636).

Each query has an explanation called a ‘preexplanation’ that is used toretain a potential part of the query's explanation should the query besuccessful. It is the explanation for the fact which is beingsubstituted into the remaining lines. It is also scanned by thecache_search routine to avoid explaining the same fact twice.

Each child query has its preexplanation explanation stored and set asthe explanation for the fact being used generate it.

The header variables for each subquery are also reduced for eachvariable that is matched to the current fact. For example if the headerquery contains the variable “a” and the queryline contains an “a”, thechild query will no longer have “a” as a query variable as this is nowsatisfied in the child query.

The value sets for the header variables so far are also preserved. Oncethe query is processed the resulting value sets, explanations and resultare also preserved.

Also within step 636 the results of each query are analysed and used toset the following values:

any_yes: true if any of the child queries returns yes

any_unknown: true if any of the child queries return unknown

Also for object queries the provisional return result is set to completebut if any of the child queries returns completeness_unknown, the returnresult is set to match.

At the end of step 626 we now have for each matching fact:

a preexplanation of the fact.

a set of results for the corresponding query and an explanation for eachset.

a return value for the query

a set of header variable values that were determined from the first line(possibly null)

Success of a child query is defined as follows:

an object query returning >0 results

a truth query returning yes

a truth query returning no when current query is a truth query and allother child queries have returned no as well.

In step 638 all duplicate sets of results are eliminated from those thatsucceeded. When duplicates are located, the result that is retained isthe one with the shortest explanation.

Control then passes to step 640 where the explanations are taken careof. This is done by merging the preexplanation for the fact with theexplanation returned by the query that returned the results. Thiscombined explanation is appended to the explanation for the main queryand associated with the returned result set by adding it and the resultset to the lists passed as parameters to the process_query call.

The final step is step 642 where the return result is calculated andreturned.

For a truth query the return result is no if all the child queriesreturned no, yes if any_yes is set and unknown otherwise.

For an object query the return result is completeness unknown ifany_unknown is true otherwise it is the result set provisionally in step634.

5.5.5 process_query refinements

Various embodiments may contain the following refinements:

5.5.5.1 Line Reordering

As discussed above, the semantics of a query in the preferred embodimentis unrelated to the order of the lines. Each line places a constraint onthe values returned and all returned objects must pass all theseconstraints.

However, the tractability of producing all the answers is affected bythe line order in embodiments which apply the constraints in the orderthey appear in the query.

For example, consider a query where the [current time][applies to] nowline appears at the end of the query and earlier lines use now intemporal partners. The query still can be viewed as making sense but anembodiment processing the lines in order would be being asked togenerate all the timepoints when a fact holds and then subsequentlyeliminate all the ones which [current time] does not apply to. As thereare an infinite number of timepoints in any timeperiod this isproblematic.

One approach in some embodiments is to simply leave it to the personwriting the query (e.g. in the translation template) to put the linesinto a sensible order.

Another approach is to add some line reordering code in theprocess_query routine where a flag is set if the current queryline ispotentially producing too many results to store and instead of justfailing, the line is reordered to the end of the query. Failure wouldonly occur if the line failed a second time (when being processed in itsnew position).

5.5.6 Example of Query Answering and Explanation Generation

FIG. 8 illustrates query processing and explanation generation in anembodiment of the invention.

The question “Is Sean Connery resident in the UK?” has been entered intoa web browser connected to an embodiment of the invention (802). Thequestion has been entered into the embodiment's “general prompt” (804).

This is a fairly challenging question to answer because the staticknowledge base contains no direct knowledge on this query. Furthermore,inferring the negative requires some difficulties because although onecan only be primarily resident in one place at a time that place can bespecified to different levels of accuracy. For example, if someone isprimarily resident in London, they are also resident in England.

However, the embodiment is able to immediately answer the question inthe negative (806) and produce a list of the static facts it used toprovide that answer (808). The key one of importance to the human useris that he has been resident in the Bahamas since at least the 15th ofMarch 1996. The static fact expressing this is:

[fact.1148017@semscript.com]  [applies for timeperiod]  [timeperiod:[timepoint: [“1996/3/15”]];  [iafter]]and the subject fact is:[fact.1148017@semscript.com]:[sean connery][is living in][the bahamas]

As the subject fact is referenced by its temporal partner, it is notlisted separately and the system translates them both together.

A small refinement is that in the translation, the system tries todistinguish between whether this date is just the earliest known datethat the fact was true or is the point when it began to be true. Withthe former case, the translation routine inserts “at least” into thetranslation (as it did in this case).

It does this by doing a query to see whether the reverse relationshipwas true at the start point of the time period (it actually communicatesslightly more knowledge than is actually in the explanation):

query f:  [sean connery]  ~[is living in]  [the bahamas] f [applies attimepoint]  [timepoint:  [“1996/3/15”]]

This query returns unknown so the “at least” is inserted. (The user whoasserted this fact sourced the knowledge from an interview that he gavedated the 15th of March 1996 where he said he was currently resident inthe Bahamas. As the date when he first started living there wasunspecified this was all that could be asserted.)

In the illustrated embodiment all the static facts are shown in theconcise explanation. Other embodiments choose not to display some of themore esoteric ones such as properties of relations, particularly if thesystem believes them to be definitely true and thus not suitable forpresentation to the user for user assessment (see section 5.10.17).

Initially the user was just presented with the answer (806), the conciseexplanation (808) and an unambiguous retranslation of the query (809).This unambiguous explanation contains a link to some text which explainshow the question was translated by reference to the translation templateand which objects were substituted (see section 5.6) However, a link forthe detailed explanation was also provided (810). By clicking on thislink the user is presented with the detailed explanation: 812 and 814(split into two for space reasons on the figure).

The detailed explanation shows every step of the solution giving all thestatic and generated facts as well as what generator was used togenerate them. This detailed explanation can be used to see exactly whatwas done in order to answer the query. In most cases the static factsare sufficient for a user to appreciate how their question was answered.(In some embodiments the names of the generators would link to profilesdescribing the generator.)

To further illustrate the knowledge generation (section 5.4) and queryanswering system, it may be helpful to explain how this question wasanswered in more detail.

The question “Is Sean Connery resident in the UK?” was translatedwithout need for further enquiry into the following query by thetranslation system (see section 5.6.6):

query [current time]  [applies to] now f:  [sean connery]  [is livingin]  [united kingdom] f [applies at timepoint] now

The process_query routine proceeds line by line as described above insection 5.5.4.

The first line is readily solved by a smart generator which generatedthe single fact:

[current time][applies to][timepoint: [“2006/7/3/11/12/02”]]

satisfying the first line. The solution for the variable now was thensubstituted into the remaining lines to produce the following query:

query f:  [sean connery]  [is living in]  [United Kingdom] f [applies attimepoint]  [timepoint:  [“2006/7/3/11/12/02”]]

This query (called in standard mode) produced “no/unknown”—i.e. there isnothing in the static knowledge base, cache or generator subsystem thatcould establish this, implying it is either false or unknown.

As the query was called in full mode, process_query then tries to answer“no” to the question by inverting the relationship and changing therelation in the temporal partner to the corresponding one as describedabove. The resulting query is:

query f:  [sean connery]  ~[is living in]  [United Kingdom] f [appliesfor all of timepoint]  [timepoint:  [“2006/7/3/11/12/02”]]

This query is then passed recursively to the process_query routine whichsets about trying to justify the first line.

There is no fact in the static knowledge base or cache that matches sothe routine goes on to try generators.

One generator whose target line matches is[generator.rightunique1@semnscript.com]

generator tp [right unique]  [applies to] r$ /s t: a$ r$ b t [appliesfor timeperiod] tp b ~[equals] c$ b ~[is a more accurate version of] c$c$ ~[is a more accurate version of] b => g: a$ ~r$ c$ * g [applies fortimeperiod] tp

This generator captures the meaning of the [right unique] attribute,essentially that if a relation is right unique and that anotherdifferent object has this relationship with the left object, then allother objects cannot have the relationship with the left object at thesame time.

The target line a$ ˜r$ c$ is successfully matched with [seanconnery]˜[is living in] [united kingdom] and the results substitutedinto the header query to get:

query tp [right unique]  [applies to]  [is living in] /s t:  [seanconnery]  [is living in]  b t [applies for timeperiod]  tp b ~[equals] [united kingdom] b ~[is a more accurate version of]  [united kingdom][united kingdom]  ~[is a more accurate version of]  b

The first three lines are easily satisfied from the following facts inthe static knowledge base. The first of which is:

[right unique] [applies to] [is living in]

This knowledge was collected from the user when [is living in] was addedusing the add_relation process (see section 5.10.7.5). As the [rightunique] value is always asked for by the process, the /s qualifier cansafely be used for efficiency as the system expects that if thisproperty holds it will be in the static knowledge base.

The next two facts matching the next two lines of the query are:

[fact.1148017@semscript.com]:[sean connery] [is living in] [the bahamas][fact.1148017@semscript.com] [applies for timeperiod] [timeperiod:[timepoint: [“1996/3/15”]]; [iafter]]

The two lines were added by someone using the add_fact process (seesection 5.10.10).

After substitution the remaining lines in the query are:

query [the bahamas] ~[equals] [united kingdom] [the bahamas] ~[is a moreaccurate version of] [united kingdom] [united kingdom] ~[is a moreaccurate version of] [the bahamas] the value of tp=[timeperiod:[timepoint: [“1996/3/15”]]; [iafter]]is retained ready for returning if the above truth query can beestablished.

The first line of this query is readily satisfied by a smart generatorcalled

[tool.equals2@semscript.com] generator => equals2@local a$ ~[equals] b$*([tool.equals1@semscript.ccm] is just the same but for positive[equals])

The tool [equals2@local] is passed the values of a$ and b$ ([thebahamas] and [united kingdom]) and simply checks that they are differentobjects. It then generates the fact:

[the Bahamas]˜[equals] [united kingdom]

The remaining two lines are satisfied with the dumb generator

[generator.geog_accuracy@semscript.com]:

generator tp a$ [is an instance of] [geographical area] f: a$ ~[isgeographically located within] b$ f [applies for timeperiod] tp => g: a$~[is a more accurate version of] b$ * g [applies for timeperiod] tpa$ gets resolved to [the bahamas] and b$ [united kingdom] turning theheader query into:

query tp [the bahamas] [is an instance of] [geographical area] f: [thebahamas] ~[is geographically located within] [united kingdom] f [appliesfor timeperiod] tp

The first line of this query is satisfied from the static knowledgethat:

[the bahamas] [is an instance of] [nation state] and [nation state] [isa subclass of] [geographical area]

The generator [generator.iaif1@semscript.com] does this inference:

generator tp,c% t: a$ [is an instance of] b /s t [applies fortimeperiod] tp b [is a subclass of] c% => g: a$ [is an instance of] c% *g [applies for timeperiod] tp

To satisfy

[the bahamas]˜[is geographically located within] [united kingdom]

the generator [geog_distinct2@semscript.com] is used:

generator t3 a$ ~[equals] b$ f1: a$ [is an instance of] c f1 [appliesfor timeperiod] t1 [geographically distinct from each other] [appliesto] c f2: b$ [is an instance of] c f2 [applies for timeperiod] t2 t3 [isthe timeperiod intersection of] [group: t1; t2] => f: a$ [isgeographically distinct from] b$ * f [applies for timeperiod] t3which captures the [geographically distinct from each other] property ofa class. As this property applies to the class [nation state] (i.e. notwo nations have overlapping territory), and as both [the bahamas] and[united kingdom] are members, and as the class is permanent it is ableto infer that they are geographically distinct for all time.

Generator [generator.geog_distinct3@semscript.com] gives meaning to therelation [is geographically distinct from]:

generator t v: a$ [is geographically distinct from] b$ v [applies fortimeperiod] t => f: a$ ~[is geographically located within] b$ * f[applies for timeperiod] t

A similar sequence happens to satisfy the second query line:

[united kingdom]˜[is a more accurate version of] [the bahamas]

so the [generator.rightunique1@semscript.com] can finally generate thefacts:

[fact12]: [sean connery] ~[is living in] [united kingdom] [fact12][applies for timeperiod] [[timeperiod: [timepoint: [“1996/3/15”]];[iafter]]

(In reality, in this embodiment, the generator will create an internalid for the generated fact that is unique. The “fact 12:” in the detailedexplanation is a simplified version of this for display to the user.)

To satisfy the final line of the original (inverted query), generator

[tool. timeperiodtotimepoint2@semscript.com] is used:

generator tr a$ [applies for timeperiod] tr =>timeperiod_to_timepoint2@local a$ [applies for all of timepoint] tp$ *which retrieves all the timeperiods that the fact applies and gives themto the [timeperiod_to_timepoint2@local] tool to see whether the providedtimepoint ([timepoint: [“2006/7/3/11/12/02”]] in this case) liescompletely within the timeperiod.

As the previous generators have provided the timeperiod [[timeperiod:[timepoint: [1996/3/15]]; [iafter]] the tool is able to generate thefact that it is true for all of this timepoint.

As this reverse query is satisfied, the process_query routine canfinally return “no”.

As discussed above the detailed explanation shown is generated as thequery is being executed, carefully retaining explanation sequences foreach fact and piecing them together to produce a single explanation ofthe answer or answers. The concise explanation is created by extractingand translating the static facts from this detailed explanation.

5.5.7 Questions about the Future

For transient facts which apply at a timeperiod ending [iafter] it isoccasionally useful to represent whether it is reasonable to assume thefact applies in the future.

For example, it is reasonable to assume that one place continues to liewithin the borders of another in the future even though this can change(change of boundaries) but it wouldn't be reasonable to say that aparticular politician will continue in the political office he or shenow enjoys in ten years time.

Whether it is reasonable to answer a query about the future with factsthat apply now and are transient is a matter of representing whetherthis assumption is reasonable or not.

One way this can be represented is to have an attribute which applies toa [fact] which says that it is reasonable to assert that it is true inthe future given a believed true [iafter] timeperiod. [fact isreasonable to assume in future] say.

The generator that determines whether a timepoint lies within atimeperiod will then behave as follows:

If (and only if), it is comparing with an [iafter] and the timepoint inquestion is in the future, it will do a query on the fact to see if theattribute applies and say it is in the timeperiod if the [fact isreasonable to assume in future] attribute is true. (In all other casesit won't.)

We can then add attributes from which this attribute can be inferred(using a generator). [relation is reasonable to assume in future] forexample, can be applied to relations which change very slowly.

For [is geographically located within] we need special treatment as theassumption varies according to whether the object is mobile or not.

This can be represented with an attribute [immobile] for objects whichcan't move and smart generators which will apply [fact is reasonable toassume in the future] for [is geographically located within] facts wherethe left object is [immobile].

The fact:

[immobile][is true of every][geographical area]

will then mean that asking whether Paris will still be in France nextyear will return “yes” but it won't for objects like [human being] wherethe object in question can move around.

5.6 Translation

Translation is the art of converting the structured knowledge andassociated entities in the system into and out of natural language.

5.6.1 Denotational Strings

A key concept to understanding various embodiments of the invention isthe concept of a denotational string. Denotational strings are stringsin a specific natural language that denote objects in the knowledgebase.

Denotational strings are linked to their corresponding objects viafacts. These facts can be stored statically or in some cases generatedby the knowledge generation system.

e.g. The facts:

[“abe lincoln”] [can denote] [abraham lincoln] [“president lincoln”][can denote] [abraham lincoln]provide a link between two ways of identifying the famous former USpresident in natural language and the internal id.

Generators can also be used to generate denotational strings. Forexample, the following generator enables a user to be able to specifyany object in the system by its identifier:

generator b% b% [is the id corresponding to the string] a$ => a$ [candenote] b% *[is the id corresponding to the string] can be implemented using thefollowing generator:

generator => string_to_id@local a% [is the id corresponding to thestring] b%

The tool string_to_id simply converts a string in the form “[<id>]” to[<id>] and creates the fact in the event that the right object isspecified and the left not; converts an id to its string form withsquare brackets around it in the event that only the left object isspecified; does nothing if neither are specified; and checks that thetwo match and generates the fact if they do, if both are specified.

This generator thus generates all facts of the form:

[“[abraham lincoln]”][can denote][abraham lincoln]

in response to any query line with the relation and at least onespecified object. This generator enables users to use any internal id tocommunicate with an embodiment.

Another example, is this generator that enables timepoints to be parsed:

generator => timepoint_parser@local s$ [can denote] [timepoint: a%] *

The tool timepoint_parser receives the string s$ (and a % if specified)and sees whether s$ corresponds to any of the various formats that weuse to specify points in time. If the string can denote one or morepoints in time the corresponding facts are generated (after comparing tosee if they match a % in the unlikely event that a % is specified).

This generator can generate facts like:

[“the 3rd of January 1992”] [can denote] [timepoint: [“1992/1/3”]][“June 1732”] [can denote] [timepoint: [“1732/6”]]

Similar generators can be written for integers and various parameteredclasses.

5.6.2 Common Translation

The common translation string is a concept which exists is variousembodiments. It is a natural short string that denotes the object innatural language. It need not be unique but needs to be fairly specificand suitable for communication about the object in context.

Common translation strings are asserted with the [commonly translatesas] relation.

An example is:

[william jefferson clinton] [commonly translates as] [“Bill Clinton”]

As with denotational strings, generators can be used to generate commontranslation strings for certain special objects such as integers,strings, timepoints etc.

5.6.3 Unique Translation

Unique recognition strings (see section 5.3.2.1) are used for uniquetranslation in the preferred embodiment.

The relation [uniquely translates as] is used to assert these.

e.g.

[william jefferson clinton] [uniquely translates as] [“William JeffersonClinton, the 42nd President of the United States”]

As with denotational strings and common translation strings, generatorscan be used to generate unique recognition strings for certain classesof object such as strings, timepoints, parametered objects etc.

e.g.

[integer: [“8128”]] [commonly translates as] [“8128”] [“hello”][uniquely translates as] [“The string ‘hello”’] [group: [abrahamlincoln]; [florence nightingale]] [commonly translates as] [“AbrahamLincoln and Florence Nightingale”]are all examples of translation facts generated by generators. The thirdexample uses a smart generator to query the knowledge base for thecommon translation strings for each object in the group and then tiesthem together in to a list.5.6.4 Relations

Translation of relations is slightly different from that of objects inthat relations are expressed using a different grammatical construction.

The preferred embodiment for the English language uses:

[is a present central form of]

to denote a string used to express the relation between two objects in amanner similar to [can denote] e.g.

[“is the spouse of”][is a present central form of][is married to]

Present central forms are also used for [commonly translates as] and[uniquely translates as]e.g.

[is married to][commonly translates as][“is married to”]

smart generators can then adapt this string with string manipulatione.g. by substituting the “is” for other words and phrases.

For example, the preferred embodiment contains the following smartgenerator:

[tool.centralpresentformconversion1@semscript.com] generator=>centralpresentformconversion@local a [is a central present formconversion of: tense$; negative$; plural$; adverb$] cpf$ *which converts central present form strings to another stringcorresponding to the supplied tense (an instance of [english tense])),negative value (an instance of [boolean] plural value (an instance of[boolean]) and adverb (an instance of [string], possibly null).

e.g.

[“have always been geographically located within”] [is a central presentform conversion of: [present perfect continuous]; [false]; [true];[“always”]] [“is geographically located within”]

Another string translation is

[is an attribute form of]

where the form in combination with the second object can be considered akind of attribute of the first object. e.g.

[“the capital of”] [is an attribute form of] [is the capital of]

That is “the capital of france” can be thought of as an attribute ofParis. This relation is useful for parsing certain types of statementsor questions. It is readily generated with a smart generator.

Another translation relation used for describing relations is:

[is a left possessive form of]

which describes the relation in a way common in English where the leftobject is perceived as belonging to the right object with a class usedto identify the relation. e.g. In English if you say:

“Paris is France's capital city”

“capital city” is a class and the possessive form of France is used toassert that the relation exists between Paris and France.

This concept is represented as:

[“capital city”] [is a left possessive form of] [is the capital of]

Again this concept is useful in translation and is prompted for in theadd_relation process (see section 5.10.7.5).

5.6.5 Other Languages

The example embodiments described give support for the English language.However the principles described herein can also be used to createembodiments which support other natural languages.

There are several thousand living languages used throughout the worldand a desirable feature in various embodiments is to provide support toeither an alternative language to English or to multiple languageseither including or not including English. As the underlying knowledgerepresentation method is distinct from natural language (unlike documentbased systems) this support can allow access to at least some of thesame underlying facts to users communicating in multiple naturallanguages.

To create support for other languages one can first create analogousrelations to those described herein for English, for each of thelanguages that one wishes to support.

For example, to support the concept of denotational strings in theFrench language one can add a relation analogous to the English language[can denote] relation.

e.g.

[“pays”] [can denote in french] [nation state]

or to support many languages consistently, a parametered relation couldbe used. e.g.

[“maa”] [can denote: [finnish]] [nation state] [“pays”] [can denote:[french]] [nation state] [“gwlad”] [can denote: [welsh]] [nation state][“pais”] [can denote: [spanish]] [nation state]

(Single language embodiments may choose to use that language for the idnaming as well.)

Similar analogous relations are needed for [commonly translates as] and[uniquely translates as]

Support is also needed for grammatical data associated with the desiredlanguage and this grammatical data could desirably be prompted for at anappropriate time in the appropriate add knowledge process (see section5.10).

For example, unlike English, many Indo-European and other languagesassociate a grammatical gender with at least some nouns and knowledge ofthis gender is necessary to produce grammatically correct productions ofthe language.

In each situation several ways of storing this knowledge should beobvious from the principles described herein.

For example:

[french masculine gender] [applies to] [“pays”]

or the gender could be associated with the denote relation, e.g.

[group: [“pays”]; [french masculine gender]] [can denote: [french]][nation state]

In addition there are sometimes irregular properties of a language thatcannot always be derived from rules. For example, in English, in thepreferred embodiment, we prompt in the add_class process (section5.10.7.3) for the plural forms of denotational strings and store this inthe static knowledge base. (English contains many irregular plurals andpluralisation in phrases can be hard to do automatically.) In languageswhere the grammatical rules contain fewer exceptions this may beunnecessary or it may be necessary to prompt for this or additionalinformation.

In summary, to support other languages requires following the principlesdescribed herein to represent any language-specific translation andgrammatical knowledge and if necessary, to prompt for it during theappropriate add knowledge processes. This knowledge is then availablefor translation when the embodiment is interacting with a speaker of thetarget language. Translation routines and templates for the languagealso need creating, again following the principles described herein butadjusted according to the properties of the target language.

5.6.6 Question Translation

In the preferred embodiment translation of natural language questionsinto queries is achieved by the use of translation templates.Translation templates contain a pattern which natural language can bematched against in combination with a description of how to translatenatural language that matches that pattern into the target entity.

This technique can translate questions into queries. The preferredembodiment can also translate natural language assertions of knowledgeinto facts (see section 5.6.10). Other techniques resolve some ambiguityissues.

As used herein, the term “question” refers to any sequence of words innatural language the purpose of which is to solicit knowledge from thesystem. It need not necessarily conform to the classic grammaticaldefinition of a question. For example, it could be in imperative formsuch as “Tell me what the capital of France is” or the meaning could beimplied. For example, in some embodiments entering just “AbrahamLincoln” could be an abbreviation for “Show me a profile screen forAbraham Lincoln”. (Generation of profile screens is discussed in section5.7.)

A translation template in the preferred embodiment contains:

the pattern: a sequence of known and unknown strings using variables forthe unknown strings;

a header query which generates results for the translation and possiblydoes other consistency checks on the strings;

a footer which is the result of the translation of the natural languagetext after values have been substituted in. This is a query whentranslating questions;

in embodiments supporting multiple natural languages, it may alsoidentify the language it supports (other embodiments may store thetemplates separately for each language making this identificationimplied).

An example translation template is:

“what is”/“what's” a b — query c,d a [is an attribute form of] c b [candenote] d — query e [current time] [applies to] now f: e c d f [appliesat timepoint] now

The top line is the template. Any sequence of three recognised stringswhere the first is “What is” or “what's” will be matched with this lineand the query at the top run to see if it produces results.

The templates are indexed by facts in the form [<string>] [is part ofthe translation] [<template name>].

When analyzing the string, we therefore only need to look at a smallnumber of templates which may match—we do not need to scan them all.

For example, if “What is the capital of France?” is asked as a questionthe engine will first attempt to recursively break this up into asequence of recognized strings. This is achieved with a function whichreturns all sequences of recognised sequences of one or more words inthe string passed to it. The function achieves this by scanning thestring from left to right. When a word boundary is found the currentlyscanned sequence of words is examined to see whether it is a recognisedword or phrase. This can be done by searching for the string in theknowledge base.

In the preferred embodiment this search for the string in the knowledgebase is done with two checks. The first to see if it is labelled asbeing part of a translation template using the query:

query [<possible substring>] [is part of the translation] a

The second check is done with the query:

query [<possible substring>] [is a denotational string pertaining to] a

This is implemented with a generator which merges all the relationsimplementing denotational strings,[generator.denotational1@semscript.com]

generator y% r [is an instance of] [english denotational relation] x$ ry% => x$ [is a denotational string pertaining to] y% *

All the relations for denotational strings are members of the class[english denotational relation] so this query will recognise anydenotational string by answering yes.

If it is recognised, it recursively calls itself with the remainder ofthe string and adds the scanned string to the start of each sequencereturned. Recognised strings can be hashed to save having to checkwhether they are recognised more than once.

For this example, one of the recognised sequences returned by thisfunction will be:

“What is” “the capital of” “France”

“the capital of” will then be textually substituted into the headerquery for a and “France” will then be textually substituted into thequery for b.

The top query will then read:

query c,d [“the capital of”] [is an attribute form of] c [“France”] [candenote] dwhich when run will return the results [is the capital city of] andd=[the nation state France].

[is an attribute form of] is a translation relation that describes howEnglish phrases can express a relation in a function sort of way. Forexample, “the spouse of”, “the mother of”, “a child of”, etc.

[can denote] is the translation relation that relates singular nouns (ornoun phrases) to an object name within the knowledge representationsystem.

The query is then run and the results will then be substituted into thebottom query as the correct translation of the question:

query e [current time] [applies to] now f: e [is the capital city of][the nation state france] f [applies at timepoint] now

This query is the correct (and only) translation of the natural languagequestion.

This query is then executed as follows:

The first line will result in a smart generator call to a tool whichwill give a single value to the variable now.

The second line will be found in the static database with e given thevalue [the french city of paris] and f given its fact name.

The final line will finally be verified by using the smart generatorwhich infers the truth of [applies at timepoint] statements from[applies for timeperiod] statements found in the static database. Thefinal line will be verified as true if the current time lies within it(or at least one of them if more than one time period is found).

The engine will then answer the natural language question with theanswer “The French city of Paris”

FIG. 9 shows the method of translating an item of natural language usingtranslation templates.

Step 902 is to break the natural language question into sequences ofrecognised substrings. Step 904 checks to see whether there are anyunprocessed sequences left, and ends the process if there are no more(or none to start with). If there are sequences still to be examined,the next one is selected (step 906) and all translation templates thatmight translate this sequence are then looked up (step 908).

Step 910 checks to see whether any of these possible translationtemplates remain and returns to step 904 if not, otherwise it proceedsto step 912 where the next unprocessed translation template is selected.Next, the current translation template is compared with the currentsequence of strings (step 914), and if they do not match then control ispassed back to step 910. (These steps ensure that every sequence ismatched with every possible translation template that might match.) Ifthey do match, step 916 is then done, and substitutions are createdbetween the variables in the template representing unspecified stringsand the strings that actually appear in the sequence. These stringobjects are substituted for those variables in the header query. Step918 which executes the query is then done. Step 920 sees whether anyresults from this query are still to be processed and if so it selectsthe next set (step 922) and substitutes the results into the translationquery to produce a possible translation (step 924). If not, it returnscontrol to step 910.

In various embodiments, question templates can also contain fields whichhelps the system translate the question or fact assertion back intonatural language. Translating back into natural language has value indemonstrating to the user that the system has correctly understood thequestion asked. In cases where the question is ambiguous, it also hasvalue in enabling the system to list various alternative understandingsof the question asked so that the user can select the one intended.

In the preferred embodiment the field is a sequence of natural languagestrings and variables resolved by the queries in the template. Totranslate the question back into natural language the system translatesthe objects into natural language and outputs the sequence ofpre-determined strings and translations to generate a translation of theentire question. In the preferred embodiment, the variables are allgenerated by a further query (equery) which generates string objectsfrom variables and objects resolved with the other queries in thetranslation. These string objects are the ones referenced in thetranslation sequence.

An example of a query with these further fields is the following:

“does” a “have a husband” — query x a [can denote] x x [is an instanceof] [human being] — query [current time] [applies to] now t: x [ismarried to] y t [applies at timepoint] now [male] [applies to] y — queryt t [uniquely translates as] x — “Does” t “have a current husband?”

If the question asked was “Does Madonna have a husband”, a match wouldbe found with the first pattern with variable a set to the string“Madonna”. The next query would be executed and as a consequence thevariable x would be set to the object denoting Madonna the person. Totranslate the question back into English, the query in the fourth fieldwould be executed giving a unique recognition string of Madonna, perhaps“Madonna Louise Ciccone, the famous singer of popular music” and thefinal re-translation of the question would be “Does Madonna LouiseCiccone the famous singer of popular music have a current husband?”

5.6.7 Resolving Ambiguity

Ambiguity is where the natural language has more than one potentialtranslation.

Ambiguity can sometimes be resolved from other information in theknowledge base. As used herein “semantic constraint knowledge” isknowledge about the meaning/use of objects in the knowledge base whichlimits how they are used by any entity that understands the object'smeaning.

Semantic constraint knowledge can be used to distinguish betweentranslations which are likely to have been intended and those which areunlikely.

Examples of semantic constraint knowledge now follow:

5.6.7.1 Left and Right Classes of a Relation

The left and right classes are properties of a relation present in someembodiments including the preferred embodiment. Left and right classesare a form of semantic constraint knowledge used in the preferredembodiment.

They are stored using facts of the form:

[<class>] [is the left class of] [<relation>] [<class>] [is the rightclass of] [<relation>] e.g. [geographic area] [is the left class of] [isthe birthplace of] [human being] [is the right class of] [is thebirthplace of]

This knowledge conveys the largest class of objects that can reasonablybe used with this relation. To put it another way, any intelligententity wanting to enquire about the existence or non-existence of arelation would not do so with objects outside these classes. (As this isthe case, the issue of whether the relation can apply to objects outsidethese classes is believed to be moot in practical use.)

As an example of how this semantic constraint knowledge can be used, inthe above “What is the capital of France?” example, the initial queryonly returned one pair of results and the question could not be made tomatch any other translation template.

However, if the initial query had produced several results: for exampleif “France” could be resolved to the name of a person, the translationwould be rejected by using facts about the right class of [is thecapital city of], e.g. the following fact (shown without a name) is inthe knowledge base:

[geographical area] [is the right class of] [is the capital of]

All the queries generated by the translation process can be subjected totype checks using such knowledge. Any line with the relation [is thecapital of] would then be subjected to checks on its right object. If itturned out that it was not an instance of [geographical area] thetranslation would be rejected.

If more than one translation remains then the engine can resolveambiguity as a last resort by asking the user for more information. Itdoes this by translating the queries back into English and listing themon the screen. The user then selects the query that he or she intendedto ask. Although individual words and phrases translating into multipleobjects are a common cause of ambiguity, different translations may alsocome from different translation templates.

[is the right class of] and [is the left class of] are permanentrelations. Furthermore, in the preferred embodiment the classes theyindicate are always permanent classes. This simplifies the ambiguityresolution as there is no need for temporal partners.

5.6.7.2 Semantic Scope of Attributes

Another related way that interpretations can be eliminated that ispresent in certain embodiments is to define a class which defines thescope of an attribute.

Attributes are shown to apply to objects with facts of the form:

[<attribute>] [applies to] [<object>]

e.g.

[unmarried] [applies to] [james buchanan]

Like left and right classes of a relation, attributes can also have aclass associated with them.

[human being] [defines the scope of] [unmarried]

Facts of this form are an additional example of semantic constraintknowledge.

This class is prompted for in the add_object process (see section5.10.7.1) when the object being added is an instance of the class[attribute].

Like left and right classes of a relation, the scope of an attribute isdefined by the semantics of the concept the attribute represents andthus provides a sanity check on any interpretation where the object isoutside this scope.

For example, when translating the question “Is Madonna single?” (seeFIG. 2) the question translation routine finds that [“madonna”] candenote two objects in the knowledge base, the singer and [the 1990 moviemadonna].

Both get translated into queries but the second translation contains theline:

f: [unmarried][applies to][the 1990 movie madonna]

As [human being] is the scope of the [unmarried] attribute and as thesystem can establish via a query that [the 1990 movie madonna] is not ahuman being, this interpretation can be eliminated as clearly notintended.

The word “single” can be automatically disambiguated here as well. Theknowledge base contains:

[“single”] [can denote] [unmarried] [“single”] [can denote] [singletrack music recording]so queries can also be generated with lines starting:[single track music recording][applies to] . . .which can be eliminated by the fact that the left class of [applies to]is [attribute] and [single track music recording] is a [class] and notan attribute.

Eliminations like this can also be efficiently done in the header query.For example, the template for this question could be:

“is”ab - query x,y a [can denote] x b [can denote] y y [is an instanceof] [attribute] - query [current time] [applies to] now t: y [appliesto] x t [applies at timepoint] now

With the y [is an instance of][attribute] line, the header query willeliminate the [single track music recording] interpretation withoutsemantic constraint knowledge even being needed.

5.6.7.3 Method for Resolving Ambiguity

FIG. 10 shows a process of testing a single translation to see whetherit can be rejected. Step 1002 sees whether there are any remaining linesin the current translation that have not yet been tested. If not, thetranslation is declared OK (step 1004) and the process ends.

If there are remaining lines, the next unchecked line is selected (step1006) and a check is made to see whether the relation in the line is avariable or a known object (step 1008). If it is a variable, control ispassed back to step 1002, otherwise a check is made to see whether theleft object is named (step 1010). If yes, the knowledge base isconsulted to see whether the allowed classes of the relation determinedby [is the left class of] facts contradict the actual class of the leftobject (step 1012). If they do the translation is rejected (step 1014)and the process ends. If the information is not there, or if the classis OK, control passes to step 1016 where a check is made to see if theright object is named. If the right object is named a check is made tosee whether the query line is a test of an attribute against an object(step 1018). If it is, a check is made to see whether the object isoutside the scope of the attribute (step 1020) and the query is rejectedif so. If it isn't a check is made on the right object against the rightclass of the relation (step 1022) and again the query is rejected if itfails (1014). If all the checks are passed, control passes back to step1002.

A process for dealing with the results of translation, includingrejecting ones that can be rejected, presenting possibilities on thedisplay, and using a fall-back strategy (see section 5.6.9 below) isillustrated in FIG. 11. Step 1102 obtains a list of possibletranslations (possibly using the process illustrated in FIG. 9 describedabove). Step 1104 tests to see whether there are any remainingtranslations and if there are not it advances to step 1112. If thereare, the next one is selected (step 1106), and it is tested to seewhether it can be rejected (step 1108). This step perhaps uses theprocess described in FIG. 10 as explained above. If it can be rejectedit is deleted (step 1110) and control returns to step 1104.

Once all the translations have been examined it proceeds to step 1112which tests to see how many translations remain. If more than onetranslation remains step 1114 is done and all the remaining translationsare displayed on screen, and the user is asked to select the intendedone (an example being illustrated in FIG. 12 and described in moredetail below). If exactly one translation remains, it is assumed to becorrect and presented as the answer (step 1116). If no translationsremain, step 1118 is done, in which the system confesses that it wasunable to translate the question and uses a fall-back strategy. Thisfall-back is described in more detail in section 5.6.9 below.

As an example, FIG. 12 illustrates how the question “When was Parisreleased?” would be dealt with by one embodiment of the presentinvention.

The system found eight translations for the string “paris” and createdqueries for seven of them. The one involving the city in France wasrejected by the translation template because the initial query asked forthe translation to be an [animated visual medium] (but it might alsohave been rejected later by checks using the semantic constrainknowledge that the left class for [was published at timepoint] has to bean [animated visual medium]). Because more than one possible translationremained, the possible results were translated back into English andpresented to the user to select from (screen 1202). The user selectedone of them by clicking on the link and the result of that selection isthe corresponding query being executed and the result displayed (screen1204). This is achieved by encoding the query as a string and passing itas a parameter in the URL using HTML GET protocol.

In contrast, with the question “Is Paris the capital of France?”(illustrated in 1206), things are different. Again, many interpretationsof the string “paris” are tried and this time the translation templatedoes not reject any of them because it is very general and works for allrelations. However, this time all the non-geographical interpretationsare rejected by analyzing the queries using the semantic constraint [isthe right class] and [is the left class] facts associated with [is thecapital of]. Here all queries containing lines similar to:

t [the 1997 movie Paris][is the capital of][france]

are rejected because the engine finds that:

[geographical area][is the left class of][is the capital of]

and proves that:

[the 1997 movie Paris]˜[is an instance of][geographical area]

The one remaining query is the one asking whether Paris the city is thecapital of France, and this one is not rejected, so the query can runinstantly without prompting the user, as only one interpretationremains. As a result the system instantly answers the question anddisplays the result (screen 1206). This instant response to the questionis also step 1116 in FIG. 11.

5.6.7.4 Rare Possibilities

A refinement found in some embodiments is to track the frequency of useof differing objects corresponding to a single denotational string anduse this data to suppress very rare interpretations. For example, acontemporary non-famous person named “Abraham Lincoln” would be entitledto have a fact saying that his name can denote him. However, it is verylikely that anyone using his name is trying to denote the former USPresident and being offered a choice every time in such circumstancescould cause irritation to users. Avoiding this can be achieved byassociating the denotational possibilities (string and object) with eachtranslation used and logging the selection when a user selects from alist of possibilities. When one denotational possibility is noticed tobe significantly less commonly used than the others (e.g. if it is theintended selection less than one in a hundred times) the embodiment canchoose to suppress it completely or relegate it to a list behind a link(e.g. saying “click here for other less common interpretations”).

5.6.7.5 Identical Answers

A further refinement extends this disambiguation strategy further byseeing whether the answers to the various questions are the same beforeprompting the user to choose between them. If the answers are all thesame, the answer is then output instead of asking the user to choose theintended question. With only a relatively small number of possibleinterpretations a further embodiment may output the answer to eachinterpretation after each interpretation instead of letting the userselect first.

Two questions having the same answer may happen by coincidence when(say) the objects being identified have the same answer. For example, aquestion asking the nationality of a person where the name entereddenotes two different people, need not ask the user which of these twopeople is meant if they both have the same nationality. Another exampleis when the question is parsed in two distinct but neverthelesssemantically similar ways. For example, the phrase “british city” withina question may be parsed as identifying a specific subclass of cities[british city] or it may be parsed as identifying members of the class[dty] with the attribute [british]. Although distinct in terms ofsemantic representation both questions are the same and will alwaysproduce the same answer.

5.6.7.6 Removing Duplicates/Equality Testing

An additional refinement present in some embodiments is to eliminateduplicate queries. Multiple translation templates may produce identicalqueries from a different way of viewing the translation. To eliminatethe duplicates when this happens involves a test for equality.

Testing queries for equality can be done with the following steps:

(1) Sorting the lines of the query into a pre-determined order(unaffected by variable names). This can be achieved by assigning allvariable names a fixed value and sorting the lines into alphabeticalorder.

(2) Normalising variable names. This can simply be done by renaming thevariables in the order they appear in the sorted lines taking variablenames from a pre-determined list. e.g. v1, v2, v3 etc. A substitutiontable is maintained so that variables that have already been renamed canhave their new name substituted in. The header variables also needlooking up and substituting from this table.

Equality is then a matter of testing for:

An identical sequence of lines.

and

The same set of header variables.

Testing equality of FACTLISTs (see below) can also done by the abovemethod but with no variables (i.e. just sorting the facts into order andtesting for equality).

5.6.8 Questions with a Premise

Some questions have an implicit assumption in them.

Examples include:

“How old was John Smith when he died?” (assumes John Smith is dead)

“How old is John Smith?” (assumes John Smith is alive)

If the assumption (or assumptions) are incorrect, methods canincorporated in some embodiments to produce an elegant answer to thequestion rather than attempting to answer it anyway.

In order to achieve this, the assumption needs to be represented andtested prior to attempting to answer the question. In some embodimentsthis can be achieved by including one or more optional “premises”queries in the question template (or elsewhere). If the questiontemplate successfully translates the question, the premises queries areexecuted first. If the query results show that the premise is false amessage can be given to the user drawing their attention to this. Themessage can be a generic message asserting the assumptions are false, aspecific message also present in the question template or a messagederived from the query.

For example, a translation template may have the following match line:

“how old was” a “when he died”/“when she died”

The header query might be:

query p

a [can denote]p

p [is an instance of][animal]

and in embodiments which don't support premises, the result of thisquery would normally be substituted directly into the translation queryeven if the entity being asked about was still alive. However, anembodiment that supports premises may have additional fields in thetranslation template enabling better behaviour. e.g.

Premise query: query [current time] [applies to] now f: [dead] [appliesto] p f [applies at timepoint] now premise message query: query q p[uniquely translates as] q premise message: q “is not dead”

The premise, premise query and premise message are additional componentsto the translation template and enable this functionality. Aftersuccessfully completing the translation template header query, thepremise query is run. If it succeeds, translation continues as normal.If it fails, the premise message query is run to generate values for thepremise message and a premise message is returned enabling the system torespond with an appropriate message.

5.6.9 Natural Language Translation Fall-Back Strategy

If the engine fails to translate the natural language text entered bythe user it can do better than simply say “Sorry”. In an exampleembodiment, the program lists all the sub-strings of the question thatit has recognised. This information gives feedback to the user about howclose the system came to understanding the question and which bits werenot understood.

Clicking on any of the strings that were recognised gives a profilescreen (see section 5.7) for the string object. The string profilescreen includes any objects that are denoted by the string. Clicking onthose gives a profile screen for the object. It is possible that astandard profile for a recognised object will answer the question thatthe user asked even though the question was not fully understood.

Other embodiments, including the preferred embodiment, use web-searchingas part of the fall-back strategy (as described in section 5.16).

Other embodiments may provide a link to the method for adding a newtranslation template (see section 5.10.16). By doing this the user hasan opportunity to ensure that his or her question and questions of asimilar format can be answered in the future.

5.6.10 Fact Translation to Natural Language

The preferred embodiment can often translate assertions of fact using amethod almost identical to the question translation described above insection 5.5.

This is achieved by the creation of an entity called a FACTLIST whichlooks a lot like a query but with no variables. A FACTLIST is simply alist of assertions of fact.

To translate assertions from natural language the template simply has aFACTLIST as the result of the translation instead of a query.

If a fact generated by the template is transient and there are notemporal partners, the preferred embodiment will then also prompt forwhen the fact is true.

If the translation produces temporal partners or the fact is permanentor true-now, the system only has to prompt for the source andconfirmation.

The use of translation templates to translate both questions and factassertions means that in the preferred embodiment, facts can be asserteddirectly from the main prompt on the site.

The ambiguity resolution techniques described above can also apply toFACTLISTs as semantic constraint knowledge applies to facts as well asquerylines. A FACTLIST can be looked at as structurally similar to atruth query.

An example translation template for fact assertions is:

a “is” b ----- query attribute, thing a [can denote] thing b [candenote] attribute attribute [is an instance of] [attribute] ----factlist attribute [applies to] thing

For more details of adding facts to the knowledge base see section5.10.10.

5.6.11 Query Translation to Natural Language

The preferred embodiment provides the user with an unambiguousretranslation of their question back into natural language which is donewithout referencing the original question provided by the user.

As seen above this enables the user to have confidence that theirquestion has been correctly understood. In the case that there areseveral interpretations of their question, it also enables the user toselect the intended one.

As described above this can be achieved by having additional fields inthe translation template that provides a query and a template into whichthe query results are inserted to produce an unambiguous retranslation.

Various other embodiments are also operable to translate a query intonatural language if these fields are absent or if the query came fromsomewhere other than being the output of the translation system.

This can be achieved with a combination of special case queries whichhave a fixed format translation in combination with a fall-backtranslation which is used when the query does not match any of thespecial cases.

For example, many queries are of the form:

query [current time] [applies to] t f: [<known left object>] [<knownrelation>] [<known right object>] f [applies at timepoint] twhich can be recognised and translated as:

“Determine whether <unique recognition string of known leftobject><unique recognition string of relation><unique recognition stringof right object> at the current time?”

The unique recognition strings can be looked up with a query.

Similar special cases can be generated with either the left object orright object unknown or when the timepoint is specified. When the leftand right objects are unknown various embodiments can refine thelanguage by checking the [left unique] and [right unique] properties ofthe relation. Other common patterns of queries can be translated bysimilar matching.

The fall-back translation can be used when the query doesn't match anyof the checked-for patterns. It may be less natural than apre-determined translation but can still be understandable. It can beimplemented in some embodiments by:

Determining the most specific likely class for each variable in thequery. This can be achieved by using the semantic constraint knowledgeto determine a class based on the variable's position within a queryline and selecting the smaller class if more than one is generated(distinct classes would imply a query that cannot be answered). Theclass will start as [object](the root class).

Giving a unique name for each variable based on the derived class, usingnumerical postfixes if the class name isn't unique (“human being 2”etc.).

Translating each line in turn using the unique recognition strings foreach known object and the variable names for each variable. The firsttime the variable is referenced using language such as “generate valuesfor human being 2 where . . . ”. In subsequent lines the language can beof the form “Limit the values for human being 2 by selecting only thosewhich . . . ”.

Capturing the meaning of the presence of query variables in the headerwith additional text mentioning the variables in the query header orgiving language corresponding to the meaning of a truth query if thereare no header variables.

5.7 Profile Generation

As used herein a “profile” is a collection of user-perceivableinformation pertaining to a specific object represented within thesystem. “Profile generation” is the facility for an embodiment of theinvention to generate profiles.

In the preferred embodiment the user perceivable information is aninformation screen delivered as a web page. It is commonly used whenusers wish to find out general information about an object rather thansomething specific (where they may choose to type a question instead).

The preferred embodiment also implements its profile generation systemby the use of multiple profile templates. Profile templates are datawhich describe the general form of a profile and, in combination withknowledge extracted from the system, enable the profile generationsystem to generate a profile for a specific object.

In the preferred embodiment a translation template exists which willtranslate a single denotational string of an object to a speciallyformatted query starting “profile:”. Queries matching this format arepassed to the profile system for rendering instead of to the queryanswering system, thereby generating an information page. This enablesusers to see a profile for an object just by typing a denotationalstring which can denote that object.

So, for example, simply typing “Abraham Lincoln” as a question willresult in the translation profile: [abraham lincoln] and the page can beimmediately displayed.

The profile generation system of the preferred embodiment includes theability to generate a profile of an object showing key information aboutthe object in a standard form. Any object within the system can be thesubject of a profile, including objects, classes, relations, facts etc.

The information shown about an object, and the format in which it isdisplayed, is a consequence of the profile template selected and theclass the object belongs to: for example, a profile of a human beingmight include information about their date of birth and occupation,while a profile of a fact might include information about when the factwas asserted and by whom.

Profiles in the preferred embodiment can contain both knowledge from theknowledge base (e.g. Abraham Lincoln's date of birth) and informationabout the knowledge base (e.g. the history of people endorsing a fact).That is, even if the implementation of the embodiment stores certainsystem specific information outside the static knowledge base theembodiment can choose to display it in a profile.

The system also allows that the same class of object may have multipletypes of profile available for different purposes. These different typesof profile may be formatted in different ways, and may also containdifferent information. For example, the ‘employment’ profile of a humanbeing might show their current and previous occupations, while the‘family’ profile of the same human being may show their parents, spouseand children.

In an alternative embodiment there may be a limit of one profile typeper class. This embodiment could still show emphasised profiles in asimilar fashion by adding classes to accommodate multiple profiles. Forexample, the family profile described above could be attached to a[human being with family] class, essentially with the same members as[human being].

In the preferred embodiment, the data about what information is includedin a particular profile and how it is formatted is encapsulated in thetemplate.

In the preferred web-based embodiment, profiles are output as HTML fordisplay to the user, but other embodiments may include output of profileinformation in any perceivable format, even including non-visual formatssuch as synthesised speech.

FIG. 13 shows an example of the profile system in operation in thepreferred embodiment. The object [abraham lincoln] is being profiledthrough several different profiles.

Screen 1302 shows him being profiled through a special profile designedspecifically for members of the class [us president] (current and formerPresidents of the United States). This is the narrowest class of which[abraham lincoln] is a member and is the default if nothing else wasspecified. This screen gives information specific to this class such asthe start and end dates of his term of office and his predecessor andsuccessors in the job.

Each profile screen contains a drop-down list of classes of which theobject is a member and which have one or more profiles attached to them(1304). In screen 1306 the user has switched the selection from “uspresident” to “human being” and is now being shown [abraham lincoln]through the default [human being] profile. In this screen US presidentrelated knowledge is absent but information common to all humans isshown, including date of birth, place of birth and marital status (themarital status fact is at death for deceased people and the current timefor live ones in this embodiment).

Where there is more than one profile available for a particular class, asecond drop-down list enables the user to navigate between profiles fora specific class (1308). In screen 1310, the user has selected the“family” profile for [human being] and the system has responded with ascreen emphasising Abraham Lincoln's family members.

Finally in screen 1312 the user has selected the profile for the class[physical object] of which [abraham lincoln] is also a member. Thisscreen contains knowledge pertaining to physical objects such as hisdate of creation (date of birth in his case) and mass. As [physicalobject] is too high up the class tree to be a principal class, thisprofile also identifies the principal class of the object being profiled(principal classes are discussed in section 5.3.4.3).

FIG. 14 illustrates how the profile system can also display knowledgestored outside the static knowledge base and how profile screens can belinked together.

Screen 1402 shows a profile screen of a single fact in the staticknowledge base. It describes the fact (1403), giving details of anytemporal partners (or subject facts) with links, gives access to userassessment (see section 5.10.17) by providing endorse (1404) andcontradict (1406) buttons, gives the status of the fact (1408) andprovides a button to immediately redo the system assessment (1409)(System assessment is described in section 5.11). It also provides anendorsement/contradiction history of the fact (1410).

Screen 1412 is a standard [human being] profile that could be obtainedby clicking on any of the links under [william tunstall-pedoe] in screen1402.

Screen 1414 is the [human being] profile with the emphasis on theircontribution to adding knowledge to the illustrated embodiment. Thissubcategory of the [human being] template is labelled “worldkb user”. Itcontains statistical information about the number of facts reported, aswell as listing recent fact assertions and assessments by this userwhich can be browsed by clicking on the link to open the relevantcorresponding profiles.

5.7.1 System Components

When a user requests a profile for an object, there are two stages tothe process:

Determine which profile template to use, based on the object requestedby the user together with optional parameters.

Expand the details of the template to produce output for display to theuser

5.7.2 Determining which Profile Template to Use

In the preferred embodiment, the choice of profile template is afunction of a particular class that the object belongs to (called the“profile class”) and a string (called the “profile type”), both of whichare optionally specified by the user. If one or both of these parametersis unspecified, the behaviour is as follows:

If the profile class is not specified explicitly, the system finds themost specific class to which the object belongs which has a profiletemplate. This is achieved (in the preferred embodiment) with thefollowing steps shown in FIG. 15

Find a list of classes of which the object to be profiled is a member(1502)

Discard any of these classes that do not have a profile templateassociated with them (1504)

Find the narrowest (most specific) of this set of classes, i.e. a classthat doesn't contain any other class in the set (1506)

Typically this process will yield only one result, but if there is morethan one the system can prompt the user to choose between thesepossibilities, or the system can choose automatically based on somedeterministic criteria (for example, choosing the most frequently usedprofile class). Other embodiments may attempt to determine which classhas the smallest number of elements.

For example, if the user requests a profile of Abraham Lincoln, andthere are profile templates available for the classes [us president],[human being], and [physical object], then the [us president] class willbe used since this is the narrowest class in the set.

If the profile type parameter is unspecified, the string “default” isused.

Alternative embodiments may use a procedure for selecting a profiletemplate that can be customised to suit a particular user.

Once a profile template has been selected, the template is expanded togenerate a profile screen to display to the user (1508)

When a profile contains transient facts, it may be that the facts inquestion do not have meaningful values at the current time because theobject in question no longer exists. The preferred embodiment deals withthis by showing a profile for the last time at which the object existed(e.g. a dead person's date of death). Other embodiments may deal withthis in various ways, including prompting the user for a differenttimepoint to generate data for, displaying a historical view of allvalues of data over the course of the object's lifetime, only displayingvalues which are applicable at the current time, or a combination ofthese techniques.

5.7.3 Expanding the Profile Template

In the preferred embodiment, profile templates are stored as XMLdocuments. The template can intersperse XHTML nodes (which have theirordinary meanings regarding formatting content) with system-definednodes (which have special behaviour associated with them).

These system-defined nodes can contain arbitrary XML data inside them(including XHTML nodes, other system-defined nodes or character data)and can carry out a variety of operations, including:

Displaying internal content verbatim

Performing processing on the internal content before displaying

Performing conditional execution of internal content

Repeating internal content a (variable) number of times

Querying the knowledge base for information

Querying other data sources (e.g. a SQL database) for information

These nodes can be combined with each other to carry out arbitrarilycomplex operations.

FIG. 16 shows the process of expanding the profile template. At thebeginning of this process, the profile template is selected as describedabove (1602).

5.7.3.1 Stage 1:

Parsing the Template

Since the template is well-formed XML, it can be parsed by a number ofthird-party libraries into an abstract syntax tree, which encapsulatesthe structure contained in the XML (1604). In the preferred embodiment,the output of this parse process is a tree structure where each node isrepresented by an object. Each node object has an (ordered) array ofreferences to child node objects, and a single reference to a parentobject. Each node object can have an arbitrary list of parameters,extracted from the node attributes in the original XML source, which canaffect the output of the subsequent processing step.

The preferred embodiment uses an object-oriented model where each nodeobject is an instance of a class document_node, or some subclass. Theclass document_node provides a method called render( ) which can beoverridden by child classes to provide special behaviour for thesenodes.

In order to distinguish between XHTML nodes (or other XML nodes with nospecial behaviour in the context of template expansion) and nodes thatrequire special processing, a namespace prefix is used. For the purposesof this document, the prefix ‘tmpl’ will be used to identify nodesrelevant to the templating system, although any prefix could be used solong as it is consistently applied.

5.7.3.2 Stage 2:

Expanding the Template

At this stage processing proceeds through the tree structure in adepth-first recursive manner, at each point executing the render()method on the node object (1606). In general, node objects will performthe render( )method on each of their children in turn, althoughparticular types of node object may override this behaviour.

For a particular node, the results of each of these render functions arecombined together (in a way that may depend on the type of the node inquestion) and returned to the caller. The value returned by the rootnode of the parse tree is the HTML document to be displayed to the user.

Therefore simply calling the render( )method on the root node of thetree implicitly causes all the nodes to be rendered recursively, and theresult of rendering the root node is the whole rendered document todisplay to the user (1608).

5.7.3.3 Example Template Expansion

FIG. 17 shows an example template expansion. Consider the exampletemplate 1702. This includes two query objects (1708 and 1710), whichfetch information from the knowledge base. It also includes two value-ofnodes (1712 and 1714), which identify places within the mark-up wherethe results of these queries will be embedded.

If the user requests a profile of an object using this template, thetemplate expansion process is carried out using the specified object(1704). The output will be HTML suitable for displaying to the user,with the corresponding values expanded (1706).

5.7.4 Template Node Class Hierarchy

5.7.4.1 document_node

All other document nodes inherit from the document_node class. Whenparsing the template, all nodes that don't have special behaviourassociated with them (including all XHTML nodes) are created asinstances of document_node.

On rendering, a document_node object concatenates and returns thefollowing:

An opening XML tag, the name of which is the tag name of thecorresponding node in the template

Attributes on the XML tag, which are just the attributes taken from theXML node in the template

The result of rendering any child nodes that are instances ofattribute_node, added as attributes to the attribute list

The result of rendering each of the child nodes that are not instancesof attribute_node, in order, concatenated together

A corresponding closing XML tag

Note that if a template were processed that consisted entirely of XHTMLnodes, they would all be instantiated as document_node objects, and theresult of this rendering process would be isomorphic to the originaltemplate document.

5.7.4.2 character_data_node (extends document_node)

The character_data_node represents character data from the XML document.Identifying which parts of the template XML to treat as character datais the job of the XML parser.

Nodes of this type are forbidden to have any child nodes (attempting toadd a child node throws an exception). During the parse phase, thecharacter data is copied from the template document.

Within the character data, values prefixed with a ‘$’ symbol indicatespecial variables, which may be expanded by the profile system to allowinformation about the environment to be passed in to the profile. Inparticular, the variable ‘$object’ will be replaced with the ID of theobject that is being profiled, which can be used both in knowledge basequeries and in text to be displayed to the user. This is seen in FIG. 17where ‘$object’ is expanded to the string [sean connery] during profileexpansion. On rendering, the content of the character data node isreturned to the caller.

5.7.4.3 query_node (extends document_node)

A query_node can carry out a query to the knowledge base or to any othersource of data (e.g. SQL database) accessible by the system on which thetemplate expansion is executing. A query_node object is instantiatedwhen a tmpl:query node is encountered in the source template.

In the preferred embodiment, this query is conceptually carried out whenthe query_node is first encountered (though execution can in fact bedelayed for optimization purposes).

Alternative embodiments allow the parameters of the query to be variedbased on the expansion of other nodes in the document.

5.7.4.4 iterator_controlled_node (extends document_node)

This is an abstract class that is used to provide common behaviour toseveral other node classes. An iterator is a pointer that runs throughvalues in a result set and executes other nodes for each value. A classthat inherits from iterator_controlled_node is one that will vary itsbehaviour depending on the presence or otherwise of an iterator that cancontrol it.

The iterator_controlled_node class has an abstract methodfind_controlling_iterator, which implements the logic for searchingthrough the page hierarchy for an iterator that controls the output ofthis node.

5.7.4.5 value_of_node (extends iterator_controlled_node)

An instance of the value_of_node class is generated by a tmpl:value-ofnode in the source XML. It is forbidden to have any child elements.

When it is encountered, it selects a value from a result set and returnsthis value as output, without any enclosing XML tags. This node selectsonly one variable from a result set: this variable is specified by theselect attribute. The query from which to select results is specified bythe “query” attribute.

The value selected from the result set may be influenced by acontrolling iterator. A value_of_node will regard another node as acontrolling iterator if it satisfies all of the following conditions:

The iterator is in the node hierarchy above the current node

The iterator is selecting from the same query as the current node

The iterator is selecting the same result variable to the current node

If a controlling iterator is found, then the node requests the currentvalue of the select variable for the controlling iterator.

If no controlling iterator is found, the value_of_node selects the valueof the variable specified in the result set specified. If there is morethan one result in the specified result set, then it will take the firstresult in this set (according to the default ordering of this resultset).

5.7.4.6 for_each_node (extends iterator_controlled_node)

The for_each_node object is generated from a tmpl:for-each node in thesource XML. It is an iterator that acts on a result set from aquery_node object.

In isolation, it will iterate over every value in the result set of thespecified query. A variable to select can also be specified, in whichcase the iterator ranges over distinct values of this variable.

An instance of for_each_node can also itself be controlled by aniterator, allowing for nested loops. A for_each_node will regard anotheriterator as a controlling iterator if it satisfies the followingconditions:

The iterator is in the node hierarchy above the current node

The iterator is selecting from the same query as the current node

The iterator is selecting a different result variable to the currentnode.

FIG. 18 shows part of an example template being transformed. Considerthe example template 1802. This template is designed to produce a listof European countries and cities within them, formatted as HTML. Whenthe template is processed, it draws from a data set (1804). A sampledata set showing a possible result of the “european_cities” query (muchreduced, for clarity) is shown at 1814.

For clarity, we shall refer to the for-each node on line 9 (1808) asiterator A, the one on line 12 (1810) as iterator B, and the one on line19 (1812) as iterator C.

When iterator A is rendered, it searches for a controlling iterator andfinds none. Therefore it uses the entire result set, and iterates overdistinct values of the variable “country”. It renders all its childnodes once for each of these three values.

During the first iteration of processing the children of iterator A,iterator B is encountered. This searches for a controlling iterator, andfinds iterator A (note that iterator C is irrelevant, since it is notabove it in the hierarchy). The current value of iterator A is‘Germany’, so iterator B filters the result set to include only resultswhere Country=‘Germany’. Iterator B then executes its child nodes forboth of these values.

The value-of node on line 13 finds iterator B as its controllingiterator, and displays the current value of this iterator each time itis executed.

After these child nodes have been executed, iterator A then carries outthe same process with each of the remaining elements in its result set.

Iterator C has no controlling iterator, so it simply iterates over alldistinct values of the “City” variable. The value-of node on line 20 iscontrolled by iterator C, and displays the corresponding value each timeit is executed.

Thus the overall effect of this template with the result set specifiedis to produce the HTML output shown at 1806.

5.7.4.7 attribute_node (extends document_node)

An attribute_node modifies an attribute on the parent node it belongsto. It is generated by a tmpl:attribute node in the template XML.

For example:

<img> <tmpl:attribute name=”src”> http:// <tmpl:value-ofquery=”image_query” select=”image_url” /> </tmpl:attribute> </img>

In this construction, the attribute_node first renders all its childnodes and concatenates the result. First, the character_data nodecorresponding to the string “http://” is rendered, and then thevalue_of_node is rendered (fetching a result from the specified query).These two results are concatenated (to produce a valid URL) and returnedto the <img> node. The <img> node sets the resultant URL string as anattribute (with the name “src”) on the node when it produces the openingXML tag. Therefore, the output might look like this:

<img src=“http://somehost.com/image.jpg”></img>

5.7.4.8 if_node (extends document_node)

An instance of if_node is created in response to a tmpl:if node in thetemplate source. The if_node allows a condition to be specified. Whenthe if_node is rendered the condition is evaluated, and if it evaluatesto true the content of the child nodes is included, otherwise the childnodes are ignored.

5.7.4.9 choose_node (extends document_node)

An instance of choose_node is created in response to a tmpl:choose nodein the template source. It acts similarly to a switch statement in C,i.e. it conditionally executes one of several branches depending onwhich condition is satisfied. The choose_node expects its children to beof type when_node or otherwise_node, and will execute the first one inthe list for which the corresponding condition is satisfied.

5.7.4.10 when_node (extends document_node)

A when_node has a condition attached to it, which has to evaluate totrue in order for the parent choose_node to execute it. The conditionattached to a when_node may be an arbitrarily complex Booleanexpression, and may include the following types of operations (amongothers):

fetching results from query_node objects

fetching the number of results in a query_node object

checking whether a query_node object returned true or false

comparing strings or numbers for equality

combining expressions with Boolean AND, OR and NOT

5.7.4.11 otherwise_node (extends document_node)

This node is equivalent to a when_node whose condition always evaluatesto true. This has the effect that the branch below this node will beexecuted if and only if none of the previous when_node conditionsevaluates to true.

5.7.4.12 macro_node (extends document_node)

A macro_node defines a section of node tree that can be repeated lateron in the document with certain parameter values expanded. The “name”attribute defines a name that will be used to denote a call to the macrolater on. The “params” attribute is a comma-separated list of parametersthat will be made available when invoking the macro later on.

Once the macro has been defined, it can be invoked by a node<tmpl:macro-name param1=“value1” param2=“value2”>, which will bereplaced in the parse tree by a copy of the nodes within the originalmacro node, with all variables replaced by their specified values.

5.8 “Answerlets”

As we have seen above, methods are described which enable the answeringof natural language questions with Yes, No or a list of one or moreentities.

Although entities can be text, this section explains how those varioussystems and methods can be enhanced to extend the class of questionsthey can answer to longer answers including the answers to (for example)“how” and “why” questions. This capability is in the preferredembodiment.

As used herein, an “answerlet” is a bit of natural languagetext—generally (but not necessarily) intended to be small, that isdesigned to give a high quality response to human questions (e.g.questions typed into a web page in a web embodiment).

In various embodiments these answerlets can be created or edited byusers of the system—e.g. internet users creating them via a webinterface.

These items of text can be stored and knowledge about them can be usedto decide when they should be displayed.

For example, the question:

Why does John Smith live in Cambridge?

can be answered by answering the query equivalent to the question:

Does John Smith live in Cambridge?

and if “yes”, examining the static facts turned up by the conciseexplanation (see section 5.5.3.2). If one or more of these has ananswerlet which is labelled as being an explanation for the fact, it isan appropriate thing to display in response to the original “why”question.

For example, in this case the answerlet could be

“[john smith 55] was a student at [cambridge university] from October1988. After graduating he remained in [cambridge uk] as he liked thecity and no work or other commitments forced him to move.”

which is identified as [answerlet: [“573”]](the answerlet itself can bestored in a separate table or in the filesystem)

Additionally, a fact exists which is of the form:

[answerlet: [“573”]][is an explanationfor][fact.1146324@trueknowledge.com]

which links the answerlet to the fact as an explanation.

[fact.1146324@trueknowledge.com] is the fact that asserts that JohnSmith is resident in Cambridge.

The example answerlet above uses IDs within the text. This is anoptional enhancement present in some embodiments. These can besubstituted for the common translations and linked to profiles.

With this embodiment, inference is fully exploited. For example, if thequestion was “Why does John Smith live in the UK?” this answerlet wouldbe printed as well, as the fact would still be included in the conciseexplanation list.

5.8.1 Multiple Language Answerlets

Additionally multiple languages can be supported by associating alanguage with each answerlet and linking answerlets together astranslations of each other.

e.g.

[written english][is the language of][answerlet: [“573”]]

[written french][is the language of][answerlet: [“1178”]]

[answerlet: [“573”]][is an accurate translation of][answerlet: [“1178”]]

In embodiments where the knowledge about the user's language skills waspresent, these can be incorporated into the process of selecting anappropriate answerlet for display.

For example, if a known bilingual French and English speaker were to aska question for which only an answerlet was available in English, theEnglish answerlet could be displayed as it was known that the user couldunderstand it.

In this situation the user could be asked if they would like to write atranslation of the answerlet which would then go in the knowledge baseas well.

Other bilingual users could post endorsements that one answerlet is asound translation of another.

Assertions that one answerlet is a sound translation of another arefacts and can be handled using user assessment in a similar fashion tothe methods described in section 5.10.17

5.8.2 Changes to Answerlets

In the preferred embodiment answerlets are fixed and changes are classedinto at least two classes.

These classes are (a) minor—where the changes do not change the meaningor content of the answerlet (e.g. grammar and spelling improvements) and(b) changes where material has been added or meaning changed.

In case (a) the new version of the answerlet is associated with the oldversion with an appropriate relation and facts about the new version areinferred from the old version as still applying. Translations of thefirst version are still considered translations of the second.

In case (b), the modified answer is considered a new answerlet and factsabout the old one need to be asserted anew if they still hold.Similarly, translations need to be created again (or modified from atranslation of an earlier version).

Which of these two classes the change goes into may originally beasserted by the person making the change. Again the relationshipsbetween the two versions of the answerlet would be a fact which can beuser-assessed using the methods described in section 5.10.17. Someautomatic verification of this could also be done. For example, systemassessment in combination with a smart generator would assert that therelationship between two answerlets was major if the number of editsrequired to transform one to the other exceeded a fixed threshold.

5.8.3 Answerlets as Word Definitions

Answerlets can also be used to answer questions which are not how or whyquestions. In fact any question where extended natural language in theanswer is desirable can be supported with this concept. For example worddefinitions can be supported with support for answerlets and facts whichrelate words to them as being a definition.

Various embodiments can support both requests for definitions of wordsgiven as a string and as a concept.

For example, the relation [is the dictionary definition of] with rightclass [object] which will work on both strings, e.g., [“pendulous”], andother objects, e.g., [pendulous]. The relation could be true now (seesection 5.3.7 for an explanation of this concept).

Questions like:

define pendulous

pendulous definition

definition of “pendulous”

what does “pendulous” mean

what is a paragraph

can then be supported with question template support as described insection 5.6.6.

5.9 User Authentication

Although querying the system can be done anonymously (e.g. profilescreen and answering natural language questions), the preferredembodiment requires the system to know who is using the knowledge basewhen changes to the knowledge base are asserted (e.g. addition ofknowledge or user assessment).

Many schemes already exist for authenticating a user of a computersystem many of which should be readily applicable to various embodimentsof the present invention. In most systems where users authenticatethemselves, they log on with a local identifier (i.e. a “username”) andthen type a secret password known only to that user.

One embodiment of the present invention uses a local identifier forusers in similar fashion. However, in some embodiments, the real-worldidentifiers within the system are used. Other embodiments combine bothschemes allowing authentication with local user entities and realentities and/or a subsequent step of linking the local entities to areal-world id. The preferred embodiment is to require a user name forknowledge addition but optionally to allow users to link their usernamewith a real [human being] entity as described herein.

The process of authentication in a particular embodiment is illustratedin FIG. 19. In order to log on to the system the user must first asserthis/her real name, identifying him/herself in the same way that that anyother object is identified (step 1902—the “select_object” process with[human being] as a parameter, described in section 5.10.6). Once thesystem has identified which entity is meant, the process checks to seewhether that entity has an associated password (step 1904). If apassword exists, the user then authenticates him/herself with thatpassword (step 1906). The system associates the user's real-worldidentifier with that session of interaction with the system. The realworld identifier is the same one as identifies the person within theknowledge base.

In an example authentication interaction with the system, the systemfirst prompts the user to say who he/she is (step 1902). The userresponds by entering his/her name (e.g. “Michael Smith”). The systemthen looks up this natural language string in the knowledge base[“michael smith”] to see which entities it could denote. If it onlydenotes one entity, the system moves immediately on to prompting for apassword (step 1906). If two or more entities in the system are denotedby this string, the system lists the unique recognition strings forthese entities and asks the user to select which entity he/she is (e.g.“Are you (1) Michael James Smith, date of birth 29th January 1969; (2)Michael R. S. Smith, the children's book author”). (See section 5.3.2.1on unique recognition strings.) This screen also has a link to follow toadd a new entity if none of the alternatives are correct (see section5.10.7.1). In the preferred embodiment, the user can also short-cut anyambiguity by entering the internal object name in square brackets (e.g.[michael James smith 32]). The square brackets show that he/she isentering an internal name and not a natural language name. The passwordentered by the user is checked for validity (step 1908), and if invalid,another opportunity given to enter the correct password (step 1910).

If the entity trying to log on to the system is not present, he/she isfirst taken through the process of adding him/herself as an object tothe system using the normal object addition sequence of screens/prompts(see section 5.10.7.1). In the preferred embodiment this is the onesituation where an unauthenticated user entity is allowed to addknowledge. The knowledge asserted is labelled as coming from the entityadded. After adding him/herself to the system, a password is promptedfor (twice to guard against the possibility of mistyping) to beassociated with this entity and used for authentication in the future(step 1912). The password entered by the user should be checked forsuitability (step 1914), and if unsuitable an opportunity given to entera better password (step 1916). The password created by the user is thenassociated with the entity in the knowledge base (step 1918).

Once the password has been entered, the user entity can be logged in(step 1920). A check is then performed to see whether or not the user isa new addition to the knowledge base (step 1922). If the entity had tobe added as a new object, his/her user rank and the time when he/shebecame a user of the knowledge base are asserted (step 1924).(Embodiments without a system of user ranks would omit this last step.)It is useful to request core facts about a new user at this stage (step1926—the “add_corefacts” process, described in section 5.10.11.1). In analternative embodiment these core facts could be gathered during theprocess of adding the entity as a new object.

When the entity denoted exists but no authentication information ispresent we have a situation where the entity has perhaps been added bysomeone else. The user must then “claim” that entity as him/herself.After warnings about the consequences of fraud, the system will promptfor a password to be used in the future (step 1912). To limit the damagefrom an impostor impersonating an individual in the knowledge base,further checks would be undertaken in the preferred embodiment such ascontacting that individual independently and/or using the true-identityestablishment techniques described below.

5.9.1 True-Identity Establishment

As used herein “true-identity establishment” is the system/methods usedto prove that the real-world identity being asserted by a user ascorresponding to him or herself truly is him or herself.

True-identity establishment is used to limit the possibility of peopleimpersonating people whom they are not and is used in variousembodiments incorporating real identity user authentication.

In various embodiments users can be given a temporary id when they firstinteract with the system and that temporary id is linked to theirclaimed identity. In this way, more than one user could potentially belinked with a real identity until the methods described herein allow oneof them to win out. This method also enables facts labelled with thetemporary id of someone who is later established to not be who they areclaiming, to be suppressed or to have a low weight associated with theiruser assessments.

The following methods are used individually or in various combinationsof these or other techniques in various embodiments. Each of thesemethods provides evidence that the user is not impersonating someonewhom they are not. Various ways of combining this evidence into anoverall belief are possible. In the preferred embodiment each item ofevidence is given a score corresponding to an estimate of the quality ofthe evidence and the user is labelled “true identity establishmentproven” once a total score threshold has been reached. Other embodimentscould use a probability based approach where each item of evidence isincorporated into a probability calculation giving an estimate of thechances they are truly who they say they are.

5.9.1.1 Linking to Documentary ID

The first method is to allow people to validate themselves using areal-world documentary id. The system can present the user with a formcontaining a unique code number which is proof that they have logged onand invite them to mail the form with a copy of a real-world id such adriver's license or passport belonging to the person they are assertingthey are. The combination of the id document together with the codenumber would be evidence that the user possessed the document sent inand thus was who they asserted they were.

5.9.1.2 Witnessing of System Use by Trusted User

An alternative method is to allow users to assert that they havewitnessed another user, whose real world identity they can vouch for,log in to the system as that user. This assertion would then tie thetrust of the second user to the first. If the user validating the seconduser was trusted or verified with other methods this would be strongevidence that the second user was who they were asserting.

5.9.1.3 Linking to Email Address and Linking Email Address to Real-WorldIdentity

Another source of information which can be used to prevent individualsfalsely claiming to be another person is by use of email addresses.

Proving that a user has access to an email address can be achieved bysending a coded link in response to a command by them when logged intothe embodiment. The coded link is sent to the address asserted asbelonging to the user entity and the email asks for it to be clicked ifthey did try to interact with the system and to ignore it otherwise.Once the website receives the click it knows the person interacting withit is likely to have access to the email address they specified. Such aprocess links the email address to the user. If the email address canalso be linked to the real-world entity asserted, a link between theperson claiming to be that entity and the entity itself can thus beproved.

This second link can be achieved in a variety of ways.

First, another user can authenticate themselves on the system and thenmake a representation that they have communicated with the real worldentity using the named email address. Real people often have long-termemail interactions with many other people. If one of them hadestablished their true identity within the embodiment and established alevel of trust, an assertion by them that this email address is linkedto this real-world identity would be strong evidence that the user iswho they say they are.

Secondly, the domain on which the email address is based may belong tothe entity or another entity closely associated with the entity. Forexample, the domain may house a website which is recognised as theofficial website of the person or their employer. Representations bytrusted users that this is the case can also be used to infer the linkbetween the real world person and their email address.

5.9.2 Non-Human Real-World IDs

Most commonly, real-world identifiers linked to users will denote humanbeings—i.e. the actual person who is logging in. However, other entitieswhich are considered capable of asserting knowledge can also besupported by various embodiments. For example, an identifier whichdenotes a business can also be used. The business would be responsiblefor limiting the authentication method (e.g. knowledge of the password)to people to whom it grants the right to represent the business inasserting knowledge. Within the invention, knowledge so asserted wouldbe labelled as being asserted by the business. The methods describedherein for true-identity establishment can be extended to non-humanidentities in similar fashion.

5.9.3 Tailoring the System to the User

By authenticating users, various aspects and behaviour of the embodimentcan be tailored to the user's preferences. Examples include:

5.9.3.1 Local Denotational Strings

By knowing who the user is, certain denotational strings can betranslated appropriately. One example of this is that the translationroutines can parse words such as “my” and “I” and successfully inferdenotational facts relating to the user entity as a result.

5.9.3.2 Privacy Management

A further advantage is in managing the privacy of users. Variousembodiments can allow an authenticated user to configure various aspectsof what personal knowledge is published for privacy and other reasons.This can be done if that user has authenticated themselves. For example,with instructions from an authenticated user, facts of the form, [emailaddress: [joesmith571@hotmail.com][is an email address of][joesmith]could be suppressed or only published to authenticated friends of[joe smith] according to the policies and selections of the user.

5.9.4 Authentication for Third-Party Systems

Once a user has established their real-world identity variousembodiments can provide an authentication service to third party systemsto prove the real-world identity associated with the user to othersystems.

One embodiment could use public key cryptography to allow the user toauthenticate themselves and then sign a message with its private key,transmitted to the third party machine which proves this. The signedmessage can contain data provided by the third party machine relating tothis session. The implementation details of various public key systemsand methods are widely known and need not be repeated here.

In an alternative embodiment, Needham-Schroeder protocol is used withthe embodiment acting as the authentication server. The details ofNeedham-Schroeder protocol have been widely published elsewhere and neednot be repeated here.

5.10 Knowledge Addition

Knowledge addition refers to the techniques by which knowledge may beadded to the system by users.

As the preferred embodiment is directed towards general internet usersadding to the system, there is no guarantee that the knowledge beingadded is correct or not added maliciously. Furthermore, most of thesegeneral internet users will not be technical or familiar with theunderlying technology.

The preferred embodiment is designed to enable almost everything neededto make the system produce and display knowledge to be added to bygeneral users including the addition of individual objects, relations,classes and attributes; the assertion of facts; and the addition ofprofile templates, generators, tools and translation templates.

For object addition and fact assertion the preferred embodiment uses anatural-language based, interrogative approach, interacting with theuser by asking natural language questions and obtaining input from theuser in response, often in an extended sequence. i.e. The knowledgeaddition subsystem can be considered in various embodiments as a naturallanguage interrogation system designed to collect real-world knowledgefrom human users for addition to the static knowledge base in structuredform.

According to various embodiments the users are interacting with at leastone remote server by feeding input into a local client computer. Aninterface is provided to the user at the user's computer which transmitsdata determined by the actions of the user (e.g. entering naturallanguage text, clicking buttons) to the remote server. Prompts and otherresponses relating to activity at the server computer are presented tothe user on the computer screen at the user's location. Despite thepossible large geographic distance between the user and the remoteserver, knowledge addition and other valuable activities involving usersare thus facilitated by actions such as providing interfaces andpresenting responses which take place locally to the user.

In the preferred web-based embodiment, the interface comprises one ormore web pages specified in HTML containing form elements. Theweb-browser on the local client computer displays the web pagecontaining instructions and form elements and actions by the user resultin data being transmitted using HTTP over the internet back to theremote web server.

5.10.1 Capturing Sources of Knowledge

According to various embodiments including the preferred embodiment, thesource of all facts in the knowledge base should be published and thusobtained during knowledge addition. This allows other users to judge theveracity of a fact by examining these sources. At a minimum an identityfor the user adding the knowledge can be recorded. In some embodimentsthis also enables automatic assessment to be done on the likely veracityof the fact.

In the preferred embodiment there are two types of source for a factasserted by a user entity. The first category of source is the userentity him/her/itself. In this case, when interacting with the system,the user asserts that the knowledge asserted is known to be truedirectly by the user (from the user's own experience). An example ofthis would be something the user has seen. In this case the user is thedirect source of the knowledge. Other valid reasons would be for factswhich are true by definition. Various embodiments could also enable auser to label themselves as the source when there are numerousindependent sources, they are certain and they are happy to takeresponsibility for the fact being true. The second category is where theuser asserts that the knowledge comes from another named source. Anexample could be a fact asserted by a book or a website or somethingcommunicated to the user by another person. In this case the user isrepresenting that the named source of the fact is the entity describedand this entity is the direct source of the knowledge. Obtaining thisinformation is a matter of prompting for it during the user'sinteraction with the system when the knowledge is being asserted. Asecond source can be identified (and if necessary added first) in thesame way that any other real-world entity is identified. The preferredembodiment also prompts the user for an optional natural languagestatement of the source of the fact. This string is also stored with thefact and can be used for later assessing of the validity of the fact byeditors and/or others.

Where the source is a named web page, the preferred embodiment takes andstores a local copy of the page. This is because a web page stored on anexternal server can be changed and the source may need to be referred tolater. If that source is cited again in a future assertion a comparisoncan be made to see whether a second snapshot of the page should bestored. In the preferred embodiment it is also legitimate to cite thelarger organisation as the source when a web page is used. For example,a fact taken from an online page of the CIA World Factbook could citethe Central Intelligence Agency as the source.

Where it is known that a single source is responsible for all pagesbelow a certain web domain, this source in combination can be inferredautomatically in the preferred embodiment. For example, the knowledgebase can contain the fact [the cia] [is responsible for content at][domain name: [“www.cia.gov”]] which would allow any document copiedfrom that website to have [the cia] automatically assigned as thesource. If there is no fact that asserts a source for all pages in thedomain, the preferred embodiment asks the user to provide one, givingthe user the option to say that there isn't one or that there is one butonly for the page cited. If the user asserts a source for that pageonly, the source and document is associated with the fact. If the userasserts a source for that domain an [is responsible for content at] factis asserted.

Other embodiments have other categories of source that are moreindirect. For example, a user could assert that a book asserts thatanother document asserts a fact. However, in the preferred embodimentknowledge is limited to either knowledge directly known by the user orasserted by a source directly known to the user. Knowledge sources thatare less direct than this are considered too unreliable.

One reason why some embodiments include indirect sources of knowledge isthat it enables it to establish confidence at least partly on the numberof independent sources of a fact that appear to exist. For example, anembodiment which labelled the source solely as the user asserting thefact could give an incorrectly high degree of confidence if a magazinemade an assertion that was then repeated by a large number ofindependent users who had read that magazine. In this case theprobability that the fact is incorrect is the probability that themagazine was incorrect, not the probability that each of the individualusers was in error. With an indirect source listed, a high degree ofconfidence can be inferred from the number of users that the magazinedid indeed assert this fact, but the confidence in the fact itself canbe assessed on the basis that there was only a single source.

It is helpful to contrast this situation with a fact asserted directlyby a large number of users, e.g. the existence of an event witnessed byten people. In this case, the chance of the fact being correct is thechance that each of these ten people was mistaken or deliberatelyasserting false information. The chance that this fact is incorrect istherefore substantially less than the situation where each of these tenis reporting a fact asserted by a single unreliable source.

A detailed method for capturing reporter, source and possibly documentinformation in some embodiments is described in section 5.10.13

5.10.2 Publication Protocols

The preferred embodiment uses a number of different protocols todetermine when and if additions by users are used widely. Otherprotocols can be used in alternative embodiments.

These protocols include:

5.10.2.1 Immediate Publication

The “immediate publication” protocol can be used for the addition of newobjects, classes and relations and permanent facts to the knowledgebase, i.e. the creation of a new id and various core facts about theobject added.

Immediate publication is also used for most assertions of facts (seesection 5.10.10) though there are exceptions (see section 5.11.2.2)

Knowledge added using this protocol is immediately available to allusers of the system. If it should prove to be wrong, user assessment(see section 5.12) enables the knowledge to be rapidlysuppressed/removed.

System assessment (see section 5.13) and fact pattern suppression (seesection 5.14) additionally help to prevent incorrect or abusive factsfrom appearing immediately (which would otherwise be dealt with usingthis protocol).

5.10.2.2 Deferred Publication

Facts added using “deferred publication” protocol are not immediatelypublished to any user other than the one who asserted them. i.e. Theyare not used in the answering of queries initiated by any user otherthan one labelled as the user who asserted them. However, they arevisible to users who specifically request a list of such facts and theseusers can use user assessment (see section 5.13) to endorse the fact.Once a number of users have endorsed the fact it becomes visible to allusers. As a fact asserted a second or more time counts as an endorsementof the original fact, it isn't a requirement that the fact can only beendorsed by users who specifically request such a list.

In various embodiments, this is implemented by endorsements andcontradictions contributing to a total score for the fact. Thedifference between facts published using deferred publication andimmediate publication is that with deferred publication, the thresholdis high enough that the assertion of the fact by the original user isinsufficient for the fact to immediately be made visible.

Deferred publication can be used for certain sensitive facts where anincorrect fact has a reasonable probability of being assertedincorrectly or maliciously and where relying on immediate publicationand later suppression by user assessment is insufficient.

The preferred embodiment uses deferred publication in just a few specialcases checked for in the system assessment system when summing theendorsements and contradictions generated by user assessment.

These cases include asserting a date of death for someone who has a dateof birth within a hundred years of the current time and when the user isnot related to the person whose date of death is being asserted (checkedfor with a query). Another example is the assertion of the end of amarriage (assertion of a timeperiod with an ending timepoint that isn't[iafter] when a timeperiod ending [iafter] is in the kb).

These examples are things which might be asserted maliciously and which,as they can become true at any time, cannot be dealt with easily usingsystem assessment or fact pattern suppression. They are also examplesthat could cause distress if they were published incorrectly.

5.10.2.3 Editor Approval

Editor (or staff) approval is where a high ranking user must firstexplicitly approve the item added before it is widely used. In thepreferred embodiment it is used for added generators, tools, translationand profile templates.

Facts published under the deferred publication protocol can also beessentially approved by high ranking users as they can also visit thelist of such facts and use user assessment to make them appear. Beinghigh ranking users, their user assessment can be configured to result inimmediate publication as the contribution to the sum that theirendorsement gives can be set to the total above the publicationthreshold in all cases. The difference between editor approval anddeferred publication is that with the “editor approval” protocol, lowranked users cannot contribute in any way to the item being published.

5.10.2.4 Trusted Users

A protocol used in some embodiments is to immediately publish all factsasserted by trusted users.

In the preferred embodiment this will happen with facts otherwisefalling under the deferred publication protocol. High ranking users havea lot to lose if they maliciously assert false facts.

5.10.2.5 Asserter Publication

Even if the knowledge is not immediately published, various embodimentscan publish the suppressed knowledge to the user who asserted it.

In an embodiment where the underlying knowledge representation system isstored in a SQL database this can be achieved by including a Booleancondition in the WHERE clause where facts are retrieved if the originalasserter of the knowledge matches the user making the query (e.g.“select . . . where asserting_user=‘[john smith 342]’ . . . )Embodiments with a separate user assessments table can do this bylinking tables.

In the preferred embodiment this is used for deferred publication offacts and additions of unapproved translation templates, generators andprofile templates. Using this technique in these latter cases allowsusers to upload and test the effects of what they are adding withoutimmediately affecting others.

5.10.3 Documentor Strings

In the preferred embodiment a documentor string may be prompted forduring the creation of an object. These are natural languagedescriptions of precisely what the object is, designed to clearlydescribe the object to a user so there can be no doubt as to its meaningand correct usage. Documentor strings are particularly useful indescribing class, relation, and attribute objects, and consequently adocumentor is always requested (though not necessarily required) duringthe creation of these types of object. Whether or not a documentorstring is requested during the creation of other types of object dependson the complexity or abstract nature of the object concerned, and theinformation about whether or not to request one is held at the level ofits principal class.

Unlike translation strings (see section 5.6.2 and 5.6.3 for moredetails) the purpose of a documentor string is to communicateinstructions to a user in natural language, rather than to be used fortranslation purposes in automatic generations of natural language.Although most physical objects can normally be identified beyond doubtby their unique recognition string, some objects, particularly classes,relations, and attributes, are more abstract and sometimes may notcorrespond to a concept that the untrained user has come across before.For example, one possible class is that of [animated visual medium]—aclass designed to cover all “movie-like” entities including televisionprograms, television adverts, movies and web-based animations. As thereis no English term for this class, it needs to be explained carefully ina documentor so that a user of the system can get a detailed descriptionwhen necessary. For example, it could be:

“This class contains all objects which are pre-recorded displays ofmoving images, e.g. movies, television adverts, flash animations.Members are not physical objects, i.e. the sequence of images isidentified, not the medium on which it may be recorded.”

This documentor of a class would typically also be displayed on theprofile screen (see section 5.7) describing the class object, i.e. theprofile screen for objects of class [class]. It can also be usedwhenever a user is using the class to add knowledge as an extra checkthey are using it correctly.

5.10.4 Processes

As used herein the term “process” denotes an interactive, automatedmethod for communication between an embodiment of the invention and auser. Most processes are designed to elicit knowledge from that user.

In the preferred embodiment, this interrogative interaction is achievedwith a sequence of web pages containing form elements, natural languageprompts and explanations and buttons. The user enters answers into theform elements and selects appropriate buttons based on the prompts.Information entered is then re-presented to the user ideally in adifferent form for confirmation. The user then has the chance to confirmwhat they said or to return and try again. Finally the knowledgeobtained from the user is added to the static knowledge base increasingthe knowledge that is known about.

Moreover, sometimes part of the process may involve another processwhich in turn may require another process etc. (termed herein as“sub-processes”). For example, when adding a new object to the knowledgebase, the user may be prompted for the name of a class to which thisobject belongs. If the user tries to specify a class which does not yetappear in the system, they may choose to add the class, opening the “addclass” process as a sub-process. Once they have finished adding theclass, the process for adding the new object needs to continue on fromwhere it left off.

5.10.5 Implementation of Processes

In one embodiment, processes can be implemented simply by coding thesequence of pages using a server-side scripting language and opening anew browser window for each sub-process. The user can then simply closethe new browser window when the sub-process is finished and return tothe original window, now able to continue.

In the preferred embodiment, the sub-process happens in a continuoussequence of pages, optionally with a single page introducing andterminating the sub-process with simple messages like “We will now beginthe process of adding this class” and “thank you for adding this class,we will now return you to where you left off”.

In the preferred embodiment all processes are coded using PHP but otherserver side scripting languages are also suitable. (A great deal ofinformation on implementing web interactions in PHP and other serverside scripting languages is described elsewhere and the details need notbe repeated here.)

When a user visits the website, an array (the “user workspace”) iscreated. This array is stored in the PHP session to make the datapersistent. One of the elements of the user workspace array is anotherarray—the “process stack”. User interaction with the system isconceptualised as a series of processes (‘select_object’, ‘add_object’,etc.). Ongoing state information for the processes is stored in theprocess stack with the current process sitting at the top of the stack.Each process is modelled as an array (the “process workspace”), itselfstored as an element in the process stack. Processes can be pushed ontothe stack and popped from it as required.

In the preferred embodiment, each process has a single controllerscript. It also has a series of pages (for user interaction) associatedwith it also written in PHP. For convenience, the files (controller andpages) for each process are stored in a separate directory belongingexclusively to that process. The controller handles which pages areshown to the user and in what order, responds according to the user'sinputs, and performs operations such as writing knowledge to theknowledge base. In the preferred embodiment pages do not make changes toprocess data directly, but may look at process data and do otheroperations solely for purposes such as determining appropriate wordingfor questions. This distinction between the relative roles of thecontroller and page scripts is not strictly necessary in terms ofproducing an implementation but was found to have some softwareengineering advantages.

5.10.5.1 Process Initialisation

A process is started by running its controller script. The controllerresumes the current session, and stores references to certain elementsof the user workspace, including the process stack, in an object (thisis a matter of convenience—other embodiments might store a copy of thewhole user workspace as an array variable, for example). The controllerneeds to know whether its process is already in existence as the currentprocess (i.e. the process at the top of the process stack), or whetherit needs to push its process onto the top of the stack as a new process.Each process has a name; the name of the current process is stored inthe user workspace, and each process stores the name of its parentprocess (the one below it in the stack) in its own process workspace. Ifthe current process name in the user workspace is the same as the nameof the process associated with the controller, then the controllerstores a reference to the current process workspace. If the processassociated with the controller is different from the current processname stored in the user workspace, then a new process workspace ispushed onto the process stack with its parent process set to the currentprocess name from the user workspace, and the current process name inthe user workspace set to the new process name.

Any values to be passed to the controller are copied into the processworkspace at this point. Parameters passed up from the parent processand return results from a sub-process are stored as arrays in the userworkspace before being copied into the process workspace and unset.Values from the process's own pages are passed to the controller as POSTor GET data.

5.10.5.2 Step Control

Navigation through the process is controlled by a series of steps in thecontroller. For the most part, a step can be thought of as a stage inthe process at which the user is asked for an input via a page. Eachstep has a name, and the process workspace includes an array of thesteps visited so far as one of its elements. This array of steps istreated as a stack, with the current step at the top. Advancing to alater step involves pushing a new step name onto the step stack, andrunning the controller until it finds the block of code corresponding tothe step at the top of the stack.

Returning from a page can be made to trigger a step advancementautomatically by including the new step value in the data POSTed fromthe page.

5.10.5.3 Back Button

In the preferred embodiment almost all pages have a back button,enabling the user to go back to the previous page. This is implementedby POSTing a value to the controller, which can be tested near the topof the controller script, and if found to be true, results in the stepstack being popped. For this reason it is important that only steps atwhich the user is shown a page are left in the step stack. (Whenever astep is executed at which no page is shown, the step stack is poppedbefore the next step is pushed onto it.)

If the back button is used on the first page shown by a process, theprocess itself is popped from the process stack, and the back buttonvalue is set as a return value for the parent process. This has theeffect of returning the user to the last page shown in the parentprocess.

5.10.5.4 Sub-Processes

If a step requires that a sub-process be called, an HTTP location headeris sent with the path to the sub-process's controller, and thecontroller script is exited. When the sub-process terminates, thecurrent controller is reloaded, any return results from the sub-processare written into the process workspace, and the script advances to thestep it was on when the sub-process was called. (It is necessary to havesome way of avoiding an infinite loop in such a step—a flag set beforecalling the sub-process or a test for a value returned by thesub-process are possible solutions.)

5.10.5.5 Process Termination

When the current process is ready to terminate, any results to bereturned are written to an array of return results in the userworkspace. Then the name of the current process stored in the userworkspace is replaced by the name of the parent process, and the processstack is popped. Finally a header redirect to the controller of theparent process is performed.

If there is no parent process (i.e. if the process being terminated isthe only process in the process stack), then a default location shouldbe specified. In the preferred embodiment process termination is handledby a method on the user workspace object, and this method has a returnpage argument which specifies the page to go to if there is no parentprocess.

5.10.6 Process for Identifying an Object (select_object)

A process that is frequently used by other processes is what is calledthe select_object process in the preferred embodiment. It enables a userto identify another object of any type. If the object is already in theknowledge base, its id is returned. If not, the user is given anopportunity to add it (using an appropriate sub-process) and then the idof the newly added object is returned.

In the preferred embodiment, all objects must have extensive naturallanguage information recorded about them as they are registered in theknowledge base, including as many denotational strings as possible and agenerally appreciated unique recognition string (see section 5.3.2.1).This enables other users to find the object (and thus the identifier);it greatly reduces the risk of a single object in the real world beinggiven two identifiers within the knowledge base, as for this to happentwo users would have to have no terms in common for what they weredenoting. For example, one internal identifier might be [abrahamlincoln]. If the following strings were registered within the knowledgebase as denoting this object “abe lincoln”, “abraham lincoln”,“president lincoln”, “president abraham lincoln”, etc., for anotherperson to miss this object they would have to list a set which had noneof these strings in common. If done properly, this is extremelyimprobable.

As discussed in section 5.3.1, in the preferred embodiment, the internalidentifiers are a natural language phrase and are distinguished fromnormal language by placing them in square brackets. This enablesexperienced users to short-cut the object selection process by simplytyping the internal identifier in square brackets. The system will thenknow that the user is directly identifying an object, and (afterchecking that the identifier exists) can skip the screen wherealternatives are listed or the unique recognition string of the objectis displayed for confirmation purposes. Other embodiments use differentsyntax to distinguish between an internal identifier and a naturallanguage string (e.g. the square brackets could be a differentcharacter). This also enables objects within the knowledge base to beidentified and readily recognised in contexts very different frominteractions with the preferred embodiment. For example, a name insquare brackets included on a printed business card or paperadvertisement can be instantly recognised as an identifier pertaining tothe preferred embodiment and users can then enter it in the system formore information, perhaps to obtain a profile screen or within a naturallanguage question. (In the preferred embodiment, such identifiers canappear and be parsed within a natural language question.) A thirdembodiment can do away with any natural language in the identifier anduse an internal identifier for objects (e.g. a unique number). Thisembodiment would rely on natural language being used to identify theobject.

FIG. 20 illustrates the process of identifying and selecting an objectin the preferred embodiment. The process begins by asking the user forthe object that he/she wishes to select (step 2002). The user may eitherenter a natural language string or the object's internal identifier ifhe/she knows it. A request is then sent to the knowledge base forobjects matching the string (step 2004).

Often this process will be initiated at a point when the class of theobject to be selected is already known (for instance, in the example ofa user asserting that “Paris is the capital of France”, the knowledgebase would expect the string “France” to represent an object belongingto the class [geographical area]—see section 5.6.7). In such cases onlyobjects with the expected class membership are sought.

Normally matches would be sought across the whole range of denotationalstrings associated with the set of objects (so that “abe lincoln”entered by the user would bring up a match with [abraham lincoln], aswell as any other people known as “Abe Lincoln”). If the string enteredby the user had the format of an internal identifier, however, thestring would only be compared with internal identifiers within theknowledge base (so that “[abraham lincoln]” entered by the user wouldonly bring up a match with [abraham lincoln], and not any otherindividual who might be referred to by the same name).

The number of matches found is examined (step 2006), and the user isgiven options accordingly. If only one match was found, the user isasked to confirm whether or not the matching object is the right one,and given alternative options, if the matching object is not what wassought, of trying again, or adding the desired object (step 2008). Ifthe user entered an internal identifier, and a match was found, then theprocess omits step 2008, and continues as though confirmation had beengiven. If no matches were found, the user is given the options of tryingagain, or adding the desired object (step 2010). If more than one matchwas found, the user is presented with the unique recognition strings ofa list of matches (each linked to their profile) and asked to select theone intended, but is also given the alternative of trying again, oradding the desired object (step 2012).

In one variant of the preferred embodiment an additional check on thenumber of matches, where more than one was found, would be carried out,and step 2012 would only be entered if the number of matches were belowsome number judged to be reasonable (otherwise the user would bereturned to step 2002 with a notice asking him/her to enter a morespecific string).

Step 2014 is a check on the user's response to the options given in step2008, 2010, or 2012. If the user opted to try again, the process returnsto step 2002. If an object was selected, the process terminates,returning that object. If, however, the user opted to add the desiredobject, a check is made to see whether the object's class is complete(i.e. labelled as having all members already fully identified in theknowledge base). If the class is complete, objects can't be added to it.This is explained to the user (step 2018), and the process returns tostep 2002.

If the object can be added, the process must first examine the class ofthe object being requested (step 2020). If the object is a class, thenthe “add_class” process is initiated (step 2022—described in section5.10.7.3). If the object is a relation, then the “add_relation” processis initiated (step 2024—see section 5.10.7.5). If, however, the objectis of any other type, a check is made to see whether the object could bea class or a relation, i.e. whether class or relation are subclasses ofthe class of the object being requested (step 2026), and, if necessary,the user is asked to clarify (step 2030). If the user's response is thatthe object is a class or a relation (step 2032), then the class is resetaccordingly (step 2034), and the process returns to step 2020. If theobject is not a class or a relation, then the “add_object” process isinitiated (step 2028—see section 5.10.7.1).

5.10.7 Processes for Adding New Objects

One type of knowledge that a user may wish to assert is the existence ofan object not already present in the knowledge base. This task may be agoal in itself, or it will come up when the absence of an identifier foran object is discovered during the assertion of other knowledge.

The act of adding a new object includes the creation of an internalidentifier for the new object, an assertion of at least one class theobject is a member of, the storage of a unique recognition string (orother unique recognition data) for the object and the collection andstorage of at least one denotational string for the object. To limit thepossibility of adding the same object twice, checks can be made on atleast the added denotational strings to see what other objects theydenote and the user is asked to confirm (by presenting their uniquerecognition data) that these pre-existing objects are not the one theuser intends to add. When the class of the added object is known,objects being denoted which are in distinct classes need not bepresented in various embodiments. For example, when adding a humanbeing, the discovery of a movie sharing a denotational string with theperson need not be prompted for as movies and human beings are distinctclasses.

Embodiments also seek to collect other useful knowledge about the newobject in the process of interacting with the user. In the preferredembodiment, adding new class and relation objects is sufficientlydifferent in terms of the knowledge collected that they are implementedin separate processes. All other objects are handled by the add_objectprocess. The add_core_facts process (section 5.10.11.1) mitigates thissomewhat by collecting additional knowledge from the user tailored tothe specific class of the object added.

5.10.7.1 Default Process for Adding an Object (add_object)

This process is for adding a new object to the knowledge base. Thisprocess is used when the object is not a class or relation as these havesufficiently different needs to use different processes (see below).add_object is used for all individual objects, physical or conceptualincluding attributes.

FIG. 21 shows the steps involved in adding a new object to the knowledgebase. The class for the object is set first (to the root class [object]by default, but can also be set to another class by a calling process.e.g. during authentication, the class can be set to [human being]).

The process begins with the user being asked for the most common termfor the object to be added (step 2102)—this will be assigned as thecommon output translation string. The knowledge base is queried forother instances of the same string within the same class, and if one (ormore) is found, the user is presented with the unique recognition stringof the corresponding object, and asked to confirm that it is not the onethat he/she is in the process of adding. The user's response is tested(step 2104)—if one of the matching objects is the intended one, anassertion is made that the string is the common output translation ofthat object (step 2105), and the process terminates returning thatobject.

Next, the process attempts to identify the principal class of the objectby consulting the ontology of the knowledge base (step 2106). Whether ornot it is able to do this will depend on the circumstances in which theprocess was called (if the class is the default root class, no principalclass will be found, but if the class has been set to [human being] then[human being] will be the principal class). If a principal class can beestablished, then it is assigned as the principal class for the object(step 2108). If the process cannot find a principal class, then theclass of the object may not be specific enough, so the “select_object”process is initiated for the user to identify and select the mostspecific class for the object (step 2110—described in section 5.10.6).The class returned by “select_object” is then tested to see whether aprincipal class can be determined from it (step 2112). If a principalclass can be determined, then it is assigned as the principal class forthe object (2108). If not, then the user is asked to confirm that theselected class really is the most specific possible (step 2114). Achange of mind at this point returns the user to the “select_object”process, but otherwise the user is permitted to continue adding theobject with no principal class.

The object's class is then tested to see whether or not it is permanent(step 2116), and if it is not then the “select_timeperiod_for_fact”process is initiated for the user to state the period of time duringwhich the object was a member of the class (step 2118—described insection 5.10.12).

The next step (2120) is to request a unique recognition string for theobject. The knowledge base is queried for any other instance of the samestring, and in the (unlikely) event that one is found, the user ispresented with the corresponding object, and asked to confirm that it isthe one that he/she is in the process of adding. The user's response istested (step 2122)—if the matching object is the intended one, all theknowledge gathered so far is asserted to be true of that object (step2105), and the process terminates returning that object. If the matchingobject is not the intended one, the user is returned to step 2120.

The process is now ready to gather a list of terms that could be used torefer to the object being added. These denotational strings areimportant in avoiding duplication within the knowledge base and totranslate as effectively as possible, so as many should be added as theuser can think of. The common output translation string and uniquerecognition string already added can themselves be regarded asdenotational strings, and are set accordingly by default. The processthen requests additional denotational strings (step 2124—illustrated indetail in FIG. 33 and described in section 5.10.9), which are checkedfor matches in turn. The addition of denotational strings may beterminated if a match is found and the user confirms that it is theobject that he/she wanted to add (step 2126). In this case all theknowledge gathered so far is asserted to be true of that object (step2105), and the process terminates returning it. Otherwise the usercontinues adding strings until he/she can think of no more.

If the object is an attribute (determined from its class), twoadditional pieces of knowledge will be required. First the user is askedto identify the attribute's scope (the most general class of objects towhich it can apply) via the “select_object” process (step 2130). Nextthe user is asked whether or not the attribute is permanent in itsapplication (step 2132). As these are the only two extra items ofknowledge required by the preferred embodiment for attributes, there isno special add_attribute process. Other embodiments may have specialhandling for other classes here or may have additional special processesfor objects of a certain type.

It is now desirable to choose an identifier for the object. The systemcreates a valid identifier from the common output translation string (tobe valid an identifier must be unique, must only contain certaincharacters, and must be within a particular range of lengths). Thisidentifier is presented to the user, who is given the choice ofaccepting it or creating a different one (step 2134). If the userchooses to create a different identifier, this is checked for validitybefore the process can continue.

Once a valid identifier has been chosen, if the principal class is onethat takes a documentor string, then the user is given the option ofadding such a string (step 2136).

If the object being added is a human being, and the user is not alreadylogged in or adding him/herself via the authentication process, then itis necessary to know whether the person being added is, in fact, theuser. The user is asked about this, if necessary, at step 2138.

During the course of the process, the assertions to be made are added toan array, either directly from the user's responses, or by inferencefrom those responses. The process is now ready to make these assertionsstoring the collected knowledge in the knowledge base (step2140—illustrated in detail in FIG. 32 and described in section 5.10.8).

Finally, the “add_corefacts” process can be initiated (step2142—illustrated in FIG. 36 and described in section 5.10.11.1). In thepreferred embodiment step 2142 is omitted if the object added was theuser him/herself (in such a case “add_corefacts” is called instead atthe end of the authentication process). The process then terminates,returning the new object's identifier.

5.10.7.2 add_object illustration

An illustrative session of a user using an implementation of theadd_object process to add the US state of Oregon to an embodiment isshown in FIG. 22 and FIG. 23.

2202 shows the initial screen of the add object process where the useris prompted for the most normal name of the object being added(additional instructions and examples are omitted for space reasons).The user enters “Oregon” and proceeds by clicking the “enter” button(2203).

2204 shows the confirmation screen for this step. Confirmation screensact as a double check against incorrectly entered information and allowthe user to change their mind and replace what they have entered. Ageneral philosophy of the preferred embodiment is that confirmationscreens should ideally re-present the knowledge given by the user in asdifferent way as possible from the way that the knowledge was initiallyprompted for, to ensure that the user fully understands the significanceof the knowledge they are providing.

2206 shows the add_object process prompting for the class of the newlyadded object. (If a class had been provided when this process was calledthis step would have been skipped.) The common translation string of theobject provided in the first step has now been incorporated into theprompt for the class. The user enters “US state” and clicks the “enter”button. The add_object process then calls the select_object process withthe string entered and, as the object being selected is a class, it isalso passed [class] as a parameter.

The knowledge base is then consulted with the query:

query a [“us state”] [can denote] a a [is an instance of] [class]which produces one result. select_object presents the one result andasks for confirmation that this is the one intended. If more than oneresult had been returned (an ambiguous denotational string) the userwould have been given the option to select the one intended.

The option to try again or add a new class corresponding to thisdenotation string is also provided.

Note that if the string provided was an unknown denotational string foran existing class the add_class process would be called but this wouldprobably not result in the addition of the class a second time as theadd_class would prompt for as many different denotational strings as theuser can think of for the “new” class. Any hits on previously existingclasses would be asked about by displaying their unique recognitionstrings linked to profiles of the pre-existing objects and if the userdiscovered at this point that the class already existed, they would begiven the opportunity to say so and add_class would exit returning thepre-existing class and adding the assertion of this string denoting theclass to the knowledge base. In this example for the class of US statesto be added twice both the user who initially added the class and theuser attempting to add it a second time would have to provide nodenotational strings in common for the second user to fail to realise itwas already in the knowledge base.

In this exemplary interaction, the user is happy to confirm that theunique recognition string for the class, “state of the United States ofAmerica” corresponds to what they were intending to say and the processproceeds to the confirmation screen 2210

In the confirmation screen 2210, the user confirms that they are indeedtrying to say that Oregon is a state and the process controller thenchecks to see whether the class is permanent or temporary with thequery:

query

[class is permanent][applies to][us state]

A permanent class is one where its members cannot cease to be memberswithout being considered something fundamentally different. As thecurrent US states were in existence prior to joining the union and couldconceivably someday leave the union and still continue to exist, theclass [us state] was considered to be a temporary class when first addedto the knowledge base. (An alternative ontology could make it permanentand consider the independent version of each state to be a differententity with a different id. In this case, this would also have been apractical approach.)

As the class is temporary the add_object process now calls theselect_timeperiod_for_fact process (section 5.10.12) to obtain a periodof time for Oregon's membership.

2212 shows the first screen in this process. The user asserts thatOregon is currently a US state.

Control then passes to 2214 where the start point for it being true isprompted for. (If the user had said it wasn't true now, the point whenit ceased to be true would also have been requested).

The user enters “Feb. 14th 1859” and the process calls theselect_timepoint process with the string which does the following query:

query tp [“Feb 14th 1859”] [can denote] tp tp [is an instance of][timepoint]

This query is answered successfully (utilising a smart generator toconvert the string into the internal representation of a timepoint) andcontrol passes to 2216 for confirmation.

The user confirms that the timepoint parsed was what they intended andcontrol then passes to 2218

On screen 2218 the user is prompted to clarify whether the date theyentered is actually the point when the relationship started or just theearliest point they know about. The user selects “no” indicating thatthis was the timepoint when membership began, enabling the process toassert the negative fact for the earlier timeperiod. Control then passesback to the add_object process.

The next screen in the add_object process is 2220. Here the user isprompted for a unique recognition string for Oregon. The user enters“the US state of Oregon”. As there is only one US state called Oregonand as everyone wanting to denote Oregon would know it was a US statethis is sufficient.

The user confirms their choice on 2302 and control passes to screen 2304

In 2304 the user is prompted to create a list of as many possibledenotational strings as possible for Oregon. 2304 continues to go aroundin a loop adding strings added by the user to the list until the userindicates that that the list is complete by clicking another button (notshown for space reasons). If any of the denotational strings can denoteany pre-existing object not in a distinct class, the unique recognitionstrings of these objects would be shown to the user for confirmationthat this is not the object they were intending to add.

The list of denotational strings is presented for confirmation on screen2306.

On screen 2308 the user is prompted for the id for the newly addedobject. The common translation string is checked for uniqueness andsuggested as a possible id for selection by the user.

2310 asks for confirmation of the id.

2312 is the final confirmation screen. It presents all the factsgathered from the interaction with the user and by default sets thesource as the user. If the user wants to communicate another sourceand/or document at this point they can do so by entering it in the addnew source box. Doing so would repaint this screen with a drop-down listnext to each fact allowing the user to change the source for one or moreof the presented facts.

When the user clicks the “add these facts” button, all the displayedfacts are added to the knowledge base, an initial user assessment by theuser and source endorsing them is added to the endorsements table and asystem assessment is done on each of these newly added facts. If any ofthe system assessments had resulted in the fact being contradicted, areport would have been displayed. In this case, all the facts were addedwithout issue so the confirmation screen 2314 is displayed.

5.10.7.3 Process for Adding Classes (add_class)

One type of knowledge that a user may wish to assert is the existence ofa new class that is not already present within the knowledge base.

The procedure in the preferred embodiment is very similar to the processfor adding any other object.

The process used in the preferred embodiment for adding a class objectis illustrated in FIG. 24. The process begins with the user being askedfor the most common term for the class to be added (step 2402)—this willbe assigned as its common output translation string. The knowledge baseis queried for other classes denoted by the same string, and if one (ormore) is found, the user is presented with that class, and asked toconfirm that it is not the one that he/she is in the process of adding.The user's response is tested (step 2404)—if one of the matching classesis the intended one, an assertion is made that the string is the commonoutput translation of that class (step 2440), and the process terminatesreturning it.

The next step (2406) is to request a unique recognition string for theclass. The knowledge base is queried for any other classes denoted bythe entered string, and if one is found, the user is presented with itsunique recognition string, and asked to confirm that it is not the onethat he/she is in the process of adding. The user's response is tested(step 2408)—if the matching class is the intended one, all the knowledgegathered so far (the common output translation string and the uniquerecognition string) is asserted to be true of that class (step 2440),and the process terminates returning it. If the matching class is notthe intended one, the user is returned to step 2406.

The common output translation string and unique recognition stringalready added can be regarded as denotational strings, and are set assuch. The process then requests additional denotational strings for theclass (step 2410), using the loop illustrated in FIG. 33 and describedin section 5.10.9. The addition of denotational strings may beterminated if a match is found and the user confirms that it is theclass that he/she was in the process of adding (step 3312). In this caseall the knowledge gathered so far is asserted to be true of that class(step 2440), and the process terminates, returning the matching class.Otherwise the user continues adding strings until he/she can think of nomore.

Plural forms are now generated for each of the denotational strings, andthese are shown to the user for correction/confirmation (step 2414).

The next step is to establish the position of the class being addedwithin the ontology of the knowledge base. The process initiates the“select_object” process and asks the user to identify and select themost specific parent class for the class being added (2416—described insection 5.10.6). If the parent class has any direct subclasses, the useris asked whether each is distinct from the class being added, or is apartial or full subset of it and this knowledge is recorded for laterassertion (step 2418). Two classes are said to be distinct if theycannot have any members in common. For example, the class of conceptualobjects is distinct from the class of physical objects and the class ofleopards is distinct from the class of trees. If a subclass is a partialsubset of the class being added, then that subclass's own directsubclasses are found, and the user is asked the same question of each ofthem. If a subclass is a full subset of the class being added, then itcan be asserted that it is a subclass of the class being added.

Refinements to this step are possible in certain embodiments. To helpthe user choose the most appropriate parent class or classes, someembodiments take the user through the ontology from a particularstarting class (for example, a parent class suggested by the user oreven the root [object] class if the user was unable to identify aparent), find the direct subclasses of that class, and ask the userwhether any of those classes is a parent of the class being added. Theuser would then be asked about the subclasses of each class to which heor she had answered ‘yes’, and this question and answer process wouldcontinue until he or she had said ‘yes’ or ‘no’ to all the possibleclasses.

Some embodiments usefully insist in the selection of just one parentclass for the class being added, but others can permit the selection ofmultiple parent classes. For example, in an ontology containing theclasses [mammal] and [sea-dwelling animal] a user could legitimately(and usefully) select both as parents when adding the class [whale].Embodiments which permit the selection of multiple parents during the“add_class” process need to check that none of the selected parents area distinct class from, or a subclass of, one of the others (it would bepointless to select [mammal] and [whale] as parents of [blue whale], andwrong to select [invertebrate] and [whale]).

The parent class is tested to see whether it is permanent or temporary(step 2420). If the parent class is temporary, then the class beingadded must also be temporary, so the process can add this fact to itsarray of assertions to be made (step 2422). If the parent class ispermanent, then the user is asked whether or not the class being addedis also permanent (step 2424). (In embodiments where there may be morethan one parent class, having any temporary class as a parent issufficient to say that the class to be added is temporary.)

The process next looks to see whether the parent class has a principalclass, i.e. is itself labelled as Principal, or is below a class whichis so labelled (step 2426). The principal class of a class's parentclass will also be the principal class of the class itself. If theparent class has a principal class, then the fact that the class beingadded is not Principal can be added to the array of assertions to bemade (step 2428). If a principal class could not be found for theparent, then the user is asked whether the class that he/she is addingcan be asserted to be Principal (step 2430). The user's response istested (step 2432), and if he/she has said that the class is notPrincipal, then a warning is given about the apparent inspecificity ofthe class, and confirmation is requested (step 2434).

The next step is to choose an identifier for the class. The systemcreates a valid identifier from the common output translation string.This identifier is presented to the user, who is given the choice ofaccepting it or creating a different one (step 2436). If the userchooses to create a different identifier, this is checked for validitybefore the process can continue.

Once a valid identifier has been chosen, the user is presented with apage (step 2438) requesting a documentor string (the user has the optionto leave this empty).

Finally the process is ready to make the assertions gathered from theuser's responses and the system's own inferences (step 2440—illustratedin detail in FIG. 32 and described in section 5.10.8). The system thenterminates, returning the identifier of the new class.

5.10.7.4 add_class illustration

An exemplary run of the add_class process is shown on FIG. 25 and FIG.26. The user is attempting to add the class of California Redwood treesto the system.

In screen 2502 the user is prompted for the common translation of theclass they wish to add. The user enters “sequoia”.

After confirmation, control goes to screen 2504 where the user isprompted for the unique recognition string for the class. The userenters “sequoia tree (the California redwood, sequoia sempervirens)”here. As this combines both common names for the species, the word“tree” and the strict latin name for the species, it is sufficient.

As with add_object all possible denotational strings are prompted for onscreen 2506. The user continues to add denotational strings and thenclicks the “no more” button when the list is complete. Potential clasheswould result in the unique recognition strings of the possible duplicateobjects being presented for confirmation but there were no clashes inthis example. Confirmation takes places on screen 2508.

Unlike with add_object, class denotational strings may need to bepluralised or recognised in their plural form. There is already a smartgenerator that can generate English plurals from one string to anotherbut for confirmation the results for each denotational string arepresented to the user and the user is allowed to correct any errors madeby the smart generator (2510). Alternative embodiments could just promptfor the plurals. These plurals are then confirmed.

On screen 2512 the immediate parent class of the newly added class isprompted for. The user asserts that a sequoia is a kind of tree.

“tree” can only denote one class so the system now displays the uniquerecognition string for this class “tree (the tall woody, perennial plantwith trunk and branches)” and asks the user for confirmation (2514).

2516 asks for confirmation of the parent class using different languagefrom the initial screen. The user again confirms that everything is OK.

In various embodiments, additional prompts attempting to firmly placethis new class within the current ontology would take place at thisstage. The knowledge base can be consulted for subclasses of theselected parent class and asked if they are a more specific parent classto the one indicated. When the parent class is fixed, each immediatechild class of the selected parent can be prompted for and the userasked whether it is possible for these classes to overlap or not. If theanswer is “no”, facts of the form class1 [is a distinct class from]class2 are generated. If “no” similar prompts are made for the immediatesubclasses of the overlapping class. By including this procedure in theadd_class process and having generators which can infer class membershipor non membership using this information, an accurate yes or no answercan be provided for membership of any object in any class.

2602 shows the screen where the user is prompted about the [class ispermanent] property of the class. As a sequoia tree cannot changespecies or be converted into any other object without being afundamentally different thing, the class is clearly permanent and theuser indicates this.

Confirmation takes places on screen 2604. Similarly to add_object 2606prompts for an id.

On screen 2608 the user is given the chance to enter a documentor forthe class (see 5.10.3).

Screen 2610 shows the final confirmation screen of the process listingall the facts gleaned from the interaction with the user. Similarly toadd_object, the user has the chance to add sources to the facts listed.

When the user clicks “add these facts” all the facts are added to theknowledge base, a user assessment endorsement added, system assessmentdone and any problems reported in an identical manner to the add_objectprocess described above.

The confirmation screen 2612 is then displayed when this has happened.

5.10.7.5 Process for Adding Relations (add_relation)

Desirable information requested about a relation includes the class ofthe objects that the relation can assert a relationship between (one foreach side) and whether the relationship is permanent or not. The classof each side of the relation can be used to resolve ambiguity inquestions. Permanence is important in knowing when the relationshipholds. Other knowledge can also be collected.

The process used in the preferred embodiment for adding a relationobject is illustrated in FIG. 27. The process begins with the user beingasked for the most common term for the relation to be added (step2702)—this will be assigned as its common output translation string. Theknowledge base is queried for other relations denoted by the samestring, and if one (or more) is found, the user is presented with thatrelation, and asked to confirm that it is not the one that he/she is inthe process of adding. The user's response is tested (step 2704)—if oneof the matching relations is the intended one, an assertion is made thatthe string is the common output translation of that relation (step2705), and the process terminates returning it.

The next step (2706) is to request a unique recognition string for therelation. The knowledge base is queried for any other relations matchingthe entered string, and if one is found, the user is presented with it,and asked to confirm that it is the one that he/she is in the process ofadding. The user's response is tested (step 2708)—if the matchingrelation is the intended one, all the knowledge gathered so far (thecommon output translation string and the unique recognition string) isasserted to be true of that relation (step 2705), and the processterminates returning it. If the matching relation is not the intendedone, the user is returned to step 2706.

The common output translation string and unique recognition stringalready added can be regarded as present central strings, and are set assuch. These are similar to denotational strings collected in add_objectand add_class. The process then requests additional present centralstrings for the relation (step 2710), using the loop illustrated in FIG.33 and described in section 5.10.9. The addition of present centralstrings may be terminated if a match is found and the user confirms thatit is the relation that he/she was in the process of adding (step 3312)after seeing the unique recognition string of the match. In this caseall the knowledge gathered so far is asserted to be true of thatrelation (step 2705), and the process terminates, returning the matchingrelation. Otherwise the user continues adding strings until he/she canthink of no more.

The process then goes on to establish the left and right classes of therelation being added. First, the process initiates the “select_object”process with a message requesting the left class of the relation (step2714—described in section 5.10.6). The object returned by“select_object” is stored as the left class. Then the processreinitiates “select_object” to request the right class (step 2716).

Step 2718 represents the collection of various core properties of therelation. First the user is asked whether the relation is permanent. Ifit isn't, a check is made to see whether the left and right classescontain objects which can have a creation date, and if this is the casefor either, the user is asked whether the object on that side of therelation must exist for facts involving the relation to be meaningful.If the left and right classes are different (and neither is a subclassof the other), then it can be inferred that the relation isantisymmetric and antitransitive, otherwise the user must be askedwhether it is symmetric and/or transitive. If the relation is transitivethen it cannot be left unique, but if it isn't transitive the user mustbe asked about the left uniqueness. If the relation is not left unique,the present central strings are checked for the presence of the definitearticle, and if it is not found, the user is asked whether the relationis “anti left unique”. (A relation such as [is a child of] is neitherleft unique nor anti left unique—“is the child of” is one of its presentcentral strings; [is a citizen of], however, is anti left unique—itwould not make sense to say that somebody is “the citizen” of acountry.) The final core property collected in step 2718 is whether ornot the relation is right unique. This involves one or two tests. If therelation is transitive, then it can be inferred that it is not rightunique. If it isn't transitive, but is symmetric, then the right uniquevalue will be the same as the left unique value. If it is neithertransitive nor symmetric, then the user must be asked about therelation's right uniqueness.

Step 2720 is the collection of left possessive strings for the relation.Often some of these strings can be generated from the present centralstrings (e.g. “child” from “is a child of”). Left possessive strings arethen requested from the user in the same way as the present centralstrings were collected, using the loop illustrated in FIG. 33 anddescribed in section 5.10.9. The addition of these strings may beterminated if a match is found, the match's unique recognition stringdisplayed (with a link to its profile) and the user confirms that it isthe relation that he/she was in the process of adding (step 2722). Inthis case all the knowledge gathered so far is asserted to be true ofthat relation (step 2705), and the process terminates, returning thematching relation.

Just as left possessive strings can be generated from present centralstrings, so new present central strings may be created from the leftpossessive strings entered by the user. If any new present central formsare created, they are shown to the user, who is given the opportunity toreject any that are wrong (step 2724).

The next step is to choose an identifier for the relation. The systemcreates a valid identifier from the common output translation string ifit is unique—adding a number to make a unique id if it is not. Thisidentifier is presented to the user, who is given the choice ofaccepting it or creating a different one (step 2726). If the userchooses to create a different identifier, this is checked for validitybefore the process can continue.

Once a valid identifier has been chosen, the user is presented with apage (step 2728) requesting a documentor string (the user has the optionto leave this empty).

The process is then ready to make the assertions gathered from theuser's responses and the system's own inferences (step 2730—illustratedin detail in FIG. 32 and described in section 5.10.8).

Once the main batch of assertions has been made, one or two more piecesof information are requested (step 2732). The first is whether a moregeneral form of the relation exists (e.g. [is married to] is a moregeneral form of [is the wife of]). The second, which is only asked ifthe relation is not symmetric, is whether the relation has anatural-sounding reverse form (e.g. [is a parent of] is the reverse formof [is a child of]). (This second question is also omitted if therelation being added is the reverse form of an existing relation.) Bothquestions are optional—the user can choose not to answer them. If eitheris answered, the user's input is sent to the “select_object” process foridentification (described in section 5.10.6). These additionalassertions are then made (step 2734—illustrated in detail in FIG. 32 anddescribed in section 5.10.8).

Finally, the system terminates, returning the identifier of the newrelation.

5.10.7.6 Illustration of add_relation

An exemplary interaction between a user and the add_relation process isshown in FIG. 28, FIG. 29, FIG. 30 and FIG. 31.

The user is wishing to add the relation linking a person with thegeographical area where they are normally resident so that factsasserting such information are supported by the system.

In screen 2802 the user is prompted the common translation of therelation they wish to add.

For both the common translation and generally appreciated uniquerecognition string various embodiments using English as the naturallanguage prompt for a present-tense expression of the relation startingwith the word “is”. This simplifies translation into other tenses as inalmost all cases, other tenses and forms can be generated just bysubstituting the “is” for other strings conforming to English grammarrules (e.g. “has been”, “is not”, “have not been”). Various embodimentsallow the user to override the insistence on this requirement andexpress the relation in other ways, prompting the user for confirmationof the other forms later in the process.

In this example, a very natural form “is resident in” accuratelyexpresses the common translation of the relation being added. The userfills this in and (after confirmation) proceeds to the next screen.

Screen 2804 shows the user being prompted for the generally appreciatedunique recognition string of the relation. In this case the only extraclarification that is needed is to clarify that this relation refers tothe general residence of the person so only a slightly augmented versionof the common translation string is entered.

Like the add_object and add_class processes, screen 2806 is where theuser provides as many alternative denotational strings for the relationas possible to maximise the chances of the relation being hit when otherusers attempt to denote it. For this screen central present forms notstarting with “is” are permitted. As with add_object and add_class thetranslation strings are added automatically to this list.

2808 is the confirmation string for these forms. The user confirms thelist and proceeds to entering the left and right classes of therelation.

The left and right classes of a relation are a consequence of thesemantics of the relation. They provide the largest class of objectswhich can appear on the left of the relation and the largest class ofobjects which can appear on the right. Any object which is not in theleft and right class cannot have the relation with any other object.

One major use of this knowledge is to disambiguate ambiguoustranslations of questions (see section 5.6.7)

2810 and 2812 prompt for the left class of the relation. In this case itis the class of human beings.

2814 and 2816 prompt for the right class. In this case it is the classof geographical areas.

This information is also useful for steering and explaining the laterstages of the process.

Many of the following screens ask about common properties of therelationship. Some of these prompts are skipped by logical deductionfrom the left and right classes so this illustration only shows some ofthe questions that may be asked. For example, the process controller caninfer that the relationship is not symmetric (a r b=>b r a) as the leftand right classes are different. It can also infer that the relationshipis antisymmetric as it can do a query and find that [human being] [is adistinct class from] [geographic area].

2902 shows the screen asking whether the relationship is permanent ornot. Some relations can change between two objects, such as this one (itis possible to cease to be resident in one place and to become residentin another place). Some relationships are permanent. The attribute beingasked about is [relation is permanent].

2904 is the confirmation screen for this step.

As [human being] is a subclass of the class [object with a creationdate] the process then enquires whether the relationship can only holdwhen the left object is in existence (in this case alive). The semanticsof some relationships require this and others do not. 2906 prompts forthis property and 2908 confirms it. The user says that this propertyholds. Note that the page uses the word “alive” on this page as it canword intelligently according to what has been entered. As it knows thatthe left class is a subclass of [biological object] it uses the word“alive” in the prompt. Otherwise the word “exists” would have been used.

2910 and 2912 do the same for the right class (rewording with the word“exists” as it can do a query to show that [geographical area] is not asubclass of animal).

2914 and 2916 prompt and confirm for the [left unique] property of therelation. This property asks whether the semantics of the relationpermit more than one entity to have the relationship with another fixedentity at the same time. As more than one person can be resident in aparticular place at any one time the answer to this question is yes(implying that [left unique] does not apply).

2918 asks about whether about the property of whether it is possible fora single entity to have the relationship despite it not being required:[anti left unique]. This property would not have been asked if therelationship was [left unique] as it could then be inferred that it doesnot apply. This property is useful with the English language fordetermining whether the indefinite article “the” can be used in denotingthe relationship. In embodiments where English is not used, this stepmight be skipped. As it is just possible for a person to be the onlyresident of a particular geographical area (a private island or smallestate perhaps), the user answers this question yes and their answer isconfirmed on 2920.

In 3002 the user is asked about the [right unique] property of therelationship. As the concept being captured is the primary residence ofa person, this relationship is [right unique] and the answer to thequestion is “no”. This is confirmed on 3004.

3006 and 3008 prompt for what in the preferred embodiment is called“left possessive forms” of a relation (using similar user interfacemethods to the prompting for denotational strings). This is analternative way of conceptualising a relationship in English and otherlanguages where the left object is thought of as being owned by theright object with a class to which the left object belongs being used tocommunicate the semantics of the relationship. For example, when we say“Paris is the capital city of France” we can also say “Paris is France'scapital city”.

In step 3010 the controller has used the left possessive forms given bythe user to suggest some other present central forms possibly missed bythe user. The articles “a/an” or “the” chosen are partly determined bythe user's responses to [left unique] and [anti left unique] properties.As both do not apply both articles are used in generating the possiblecentral present forms.

After the user confirms these forms control passes to screen 3012 wherethe user is prompted for an id in a similar manner to the add_object andadd_class processes.

3014 is where the user is prompted for a documentor. Documentors areparticularly important for relations.

3102 is where the collected facts are presented to the user andalternative sources can be specified. This is similar to thecorresponding steps in add_object and add_class. When the user confirmsthese, the facts are written to the static knowledge base and systemassessed as with the other add knowledge processes.

When this is completed without anything to report, control passes to3104 where the user is asked about a more general form of the relationjust added. This knowledge can be used to generate more general forms ofa relation from a more specific fact stored in the static knowledgebase.

The user says they cannot think of one at the moment and control passesto screen 3106.

In 3106 a reverse form is asked about. A reverse form is a semanticallyidentical relationship where the left and right objects are reversed. Inthis case, the reverse form of the relation was already present in theknowledge base. If it was not, the add_relation process would have beenrepeated for the reverse form (and by passing the name of the reverserelation to the process it would be able to skip many steps where theanswers could be logically inferred from the properties of the relationwhich were the reverse, i.e. the left and right classes and properties).

When a reverse relation is specified in add_relation, the preferredembodiment labels the more newly added relation with the property[reverse form preferred]. This property is used by add_fact and thequery processing system to switch around relations which have thisproperty by changing them for their reverse relation and swapping theleft and right objects. For add_fact this keeps the static knowledgebase “tidy” by not having semantically identical facts in two formats(e.g. having <attribute>[applies to] <object> facts as well as<object>[is]<attribute> facts). For query processing it also means thatthe generator which generates reverse forms can be ignored, gaining someefficiency. Alternative embodiments which have the generator active andallow static facts to be asserted both ways around are also believedpractical though.

Screens 3108, 3110 and 3112 show the reverse form of the relation beingselected and confirmed and the reverse relationship being confirmed.Finally the addition of the initial relation is confirmed (3114).

5.10.8 Assertions Process/Routine

At the end of each of the “add_object”, “add_relation”, and “add_class”processes the user is presented with a list of the assertions to bemade, and given options to associate different assertions with differentsources. These assertions must be confirmed before they can be made. Themethod by which this is done is illustrated in FIG. 32.

The first step (3202) is to loop through the array of assertionschecking that each is permissible. If any are not permissible (forexample, if one of the denotational strings supplied by the user todenote a particular person is suppressed for knowledge addition), thenthe list of assertions is shown to the user (step 3204) with the problemassertions highlighted. The user is asked to correct the problems.Continuing from this point will take the user back to the stepassociated with the problem assertion (step 3206)—if there is more thanone problem assertion, then the user is taken back to the earliest onein the process.

If all the assertions are permissible, then the user is shown them as alist, together with the source for each (step 3208). By default thesource is the user him/herself. This page gives the user options to adda new source to the available sources (by entering the name of thesource in an input box), confirm the assertions as presented, or changea particular assertion. If more than one source is available, he/she canassociate particular assertions with particular sources beforeconfirming. The user's response is then tested (step 3210).

If the user chose to change an assertion (‘disagree’), then he/she istaken back to the step associated with that assertion (step 3206).

If the user chose to add a new source, the string input must beidentified as a source, and, if possible, an animate source identified(step 3212). The method for doing this is illustrated in detail in FIG.38 (described in 5.10.13). Once the source has been identified, it isadded to the list of sources available (step 3214). The user is returnedto the assertions confirmation page (step 3208). Users can add as manysources as desired (one at a time) by looping through steps 3208 to3214.

Once the user is ready to confirm the assertions (and has associatedsources with them), a check is made on whether or not the user is loggedin (step 3216). If the user is not logged in, he/she is required to doso (step 3218—the “authenticate” process illustrated in FIG. 19 anddescribed in section 5.9. The assertions can now be made (step 3220).

5.10.9 Denotational Strings Collection

Denotational strings are related to their object by various relations,including [can denote], as shown in examples in section 5.6.1. They arenames or phrases which may be used to denote the object, and areimportant in translating user queries and in avoiding the addition ofduplicate objects to the knowledge base. In the preferred embodiment thesame method for gathering these strings is used by the “add_object”,“add_relation”, and “add_class” processes. This method is illustrated inFIG. 33.

First, a page is presented to the user, requesting a name or phrasewhich could be used to denote the object being added (step 3302). Thepage also gives options to delete an already added string, or to stopadding strings. The user's response is checked at step 3304.

If the user chooses to add a new string, the knowledge base is queriedfor objects which can also be denoted with that string and which may bethe object the user is intending to add, in some embodiments this checkmay involve verifying that the possible matching object is not adistinct class from any known class of the object being added (step3306). If there are no matches, the string is added (step 3308), and theuser is taken back to step 3302. If one (or more) matches is found, theuser is presented with the unique recognition strings for thecorresponding objects, and asked to confirm that it is not the one thathe/she is in the process of adding (step 3310). The user's response istested (step 3312)—if one of the matching objects is the intended one,the loop ends, and that object is returned (step 3314). If the user issure that the matching object is not the one being added, the string isadded (3308), and the user is taken back to step 3302.

If the user's response at step 3304 is to delete a string, the string isdeleted (step 3316), and the user is taken back to step 3302.

If the response at step 3304 is to stop adding strings, the user isshown a list of the strings he/she has added, and asked to confirm thatthey can all be used to denote the object (step 3318). The response tothis message is tested (step 3320). If the user won't confirm, thenhe/she is returned to step 3302 (where any problem strings can bedeleted). When the user is happy with the list of strings to beassociated with the object, the loop ends, and the list of strings isreturned (step 3322).

In the preferred embodiment, a count is kept of how frequently eachdenotational string is used by users of the system to denote an object.These counts can be used to present denotational strings representing anobject in order of popularity when displaying (say) a profile of theobject. The preferred embodiment also keeps a count of how frequentlyeach ambiguous denotational string is used to denote each of thepossible objects it may refer to. In situations where one object is manytimes more common than another (e.g. a celebrity and a much less famousperson with the same name), it can be used in some embodiments to assumethat the more frequent choice is intended, thereby saving each user fromhaving to choose every time. In the preferred embodiment it is also usedto list ambiguous translations in order of likelihood.

5.10.10 Process for Adding Factual Knowledge (add_fact)

One of the desirable actions that a user can perform is to assert newfactual knowledge. In the preferred embodiment this is that a namedrelationship exists between two named entities, and, in the case of anon-permanent relationship, when that relationship exists (i.e. they arealso prompted for a timeperiod). In the preferred embodiment, negativerelationships can also be asserted.

Again, this is achieved by a sequence of prompting screens presented tothe user asking for the fact and (if necessary) the timeperiod. Theentire assertion is translated into natural language using uniquerecognition translations for confirmation prior to being recorded in theknowledge base. If any object or relation is missing during this processthe system will digress to prompt for the information necessary to addthis entity to the knowledge base.

The process by which relationships are asserted is illustrated in FIG.34. The process begins (step 3402) by requesting the fact in naturallanguage (“as you would tell it to another person”). The system attemptsto parse the string entered by the user (step 3404). If it can beparsed, the elements of the fact (at least left object, relation, andright object, but possibly also negativity and temporal information) areextracted from the string (step 3406). Translation is described insection 5.6.10. If the user's string is not understood, then the user ispresented with a page (step 3408) where the left object, relation, andright object are entered as separate elements along with detailedexplanation and examples. A check is made that all three elements havebeen entered (step 3410)—once they have, the process can continue.

The next stage is to identify each of the three fact elements. First,the relation is sent for identification by the “select_object” process(step 3412—described in section 5.10.6). Once the intended relation hasbeen established, the left and right classes of the relationship arefound (step 3414).

The first object is then sent for identification by the “select_object”process (step 3416)—it is sent with the left class of the relationshipas a parameter to ensure that “select_object” only looks for relevantobjects. Next, the second object is sent to “select_object” with theright class as a parameter (step 3418).

A translation of the fact is created (using the unique recognitionstring of each element) and shown to the user for confirmation (step3420). The user's reaction is tested (step 3422). If the user does notagree that the fact as stated is the fact that he/she is intending toadd, then the process returns to the beginning.

If the user confirms the fact translation, the process continues bytesting whether or not the fact is a permanent one, and acts accordingly(step 3424). If it is not inferred to be permanent, a timeperiod for thefact is requested using the “select_timeperiod_for_fact” process (step3426—described in section 5.10.12).

Next it is necessary to establish the source (and preferably an‘animate’ source) for the assertion that is about to be made. Under somecircumstances (if the process is called when a user is addinghim/herself to the knowledge base during authentication), the source forthe assertion might already have been set as the user. The process teststo see whether the source is the user (step 3428). If the source is theuser, then he/she can be attributed as the animate source for theassertion (step 3430). If the source is still unknown, the user is askedto specify a source (step 3432). The user can state that he/she is thesource, or provide a different source perhaps a named individual or workof reference, or the URL of a web document. The user's response isexamined (step 3434). If the user has stated that he/she is the source,a check is made to see whether he/she is logged in (step 3436). If not,he/she is required to log in (step 3438—the “authenticate” processillustrated in FIG. 19 and described in section 5.9). Once the user'sidentity has been established, he/she can be attributed as the animatesource for the assertion (step 3430). If the user is not the source ofthe fact, then the specified source must be identified and an attemptmade to establish an associated animate source (step 3440—illustrated indetail in FIG. 38 and described in section 5.10.13).

Finally (step 3442), the relationship, the source of the assertion, and(if relevant) any timeperiods are asserted. If the fact is already knownto the knowledge base, then this assertion will count as an endorsement.

Facts can also be parsed from complete natural language assertions typedby the user into the main system prompt in some embodiments (e.g. “Parisis the capital of France”). If the translation system translates thisinto an assertion of a fact the add_fact process can be started at step3406 exactly as if the initial assertion had been typed into the promptcorresponding to step 3402

See section 5.6.10 for more details on how these are translated. Whenthis method cannot correctly parse the fact being asserted, the methoddescribed above can be used as a fall-back.

5.10.10.1 Partially Pre-Specified Fact Addition

In the preferred embodiment, the add_fact process can be called with oneor more of the three objects already filled in.

An example of where this is useful is on profile screens where knowledgeis unknown. For example, on a person profile there may be a slot for“place of residence”. The profile was unable to get this knowledge fromthe knowledge base so instead creates a link with the person's id, therelation [is resident in] and a blank. A user visiting the profile whoknows the place of residence can then simply click on the link to betaken

An example of this is illustrated in FIG. 35.

A user has typed “Victoria the Empress of India” into the general promptthe system has translated this into a request for a profile screen forthe historical figure[queen victoria] and displayed the default profilewhich is the default [human being] profile (3502).

One of the slots on this profile is for the subject's place of birth.The query that was designed to display this information returned“unknown” so the template created a link which would allow the user tospecify it (3503). The link points to the add_fact controller scriptwith parameters encoded specifying the two known elements of the fact tobe added.

add_fact recognises these parameters and instead of prompting for allthree elements, only prompts for the one that is missing (3504). (Otherembodiments may use the left or right class of the known relationship tophrase the prompt better, e.g. “What geographical area is the birthplaceof Queen Victoria of the United Kingdom of Great Britain and Ireland?”.)The user enters “London”.

add_fact uses select_object to locate the correct entity and asks forconfirmation of the fact to be added (screen 3506). After sourceselection and confirmation of the fact being added the user opens theprofile again (3508). This time the knowledge is in the knowledge baseand the profile correctly shows her place of birth.

5.10.11 Core Facts

According to various embodiments with the principal class concept, eachprincipal class has certain core facts associated with it. This isknowledge which varies between members of the class and which isconsidered important. When a new object is added to the knowledge base,the preferred embodiment will also prompt the user for the core factsassociated with the principal class of the object. For example, with theprincipal class [human being], the preferred embodiment will prompt forthe sex of the object (person) added and the date of birth.

In alternative embodiments, core facts are associated with any class andinstead of prompting for the core facts associated with the principalclass of the object, a search is made for the most specific class whichhas core fact information associated with it.

5.10.11.1 Process for Adding Core Facts (add_core_facts)

FIG. 36 shows the steps involved in adding core facts about an object.First of all, it is necessary to establish whether or not any core factsare associated with the object's principal class (step 3602). If no corefacts are so associated, the process ends. Otherwise, an array of thecore facts is created (step 3604), and a loop is entered between step3606 (which requests the answer to each core fact in turn) and step 3612(which checks to see whether any core fact questions remain to beasked).

Once a core fact question has been asked (3606), the answer is sent foridentification by the “select_object” process (step 3608—described insection 5.10.6). The user's answer can then be combined with the currentobject and the relation relevant to the core fact to make an assertionusing the “add_fact” process (step 3610—described in section 5.10.10).When no more core fact questions remain to be asked (3612), the processterminates.

5.10.12 Selecting a Time Period (select_timeperiod_for_fact)

Many relationships in the knowledge base require a temporal partner.Consequently, a user wishing to assert a temporary relationship must beprovided with a mechanism to identify the relevant time period for thatrelationship. In the preferred embodiment the process of selecting a[timeperiod] object is called whenever the user directly asserts arelationship of a temporary nature (such as [is married to]), orwhenever an object is added which is a member of a class defined asbeing temporary (such as [politician]).

FIG. 37 illustrates the process in the preferred embodiment of selectinga time period. The user is first asked whether the fact is true now(step 3702), and is then asked for the earliest time when the fact wastrue (step 3704). The “select_object” process is initiated with thestring entered by the user and the class [timepoint] as parameters (step3706—“select_object” is described in 5.10.6). “select_object” returns a[timepoint] object.

If the user has said that the fact is true now (step 3708), the secondtimepoint will be [after] (step 3710), but if the fact is not true now,the user is asked for the latest time when the fact is true (step 3712).As before, the “select_object” process is initiated with the stringentered by the user and the class [timepoint] as parameters (step 3714).

After the second [timepoint] object has been established, a check ismade (step 3716) to see that the timepoints make a reasonable timeperiod (the second must be later than the first). A problem encounteredat step 3716 results in the user being shown an explanatory message anda request to enter the initial timepoint again (step 3718).

Once a reasonable time period has been established, it is useful toascertain whether there is also a prior time period (when the fact isnot true). For example, if a user wishes to assert that two individualsare married, and is able to give a date for the marriage (howeveraccurately), then it is reasonable to assert at the same time that thecouple were not married for all time before that starting timepoint. Analternative embodiment would omit this stage, and proceed straight tothe creation of the [timeperiod] object at step 3732. In the preferredembodiment, however, a check is made on the starting timepoint (step3720). If it is [time zero] or [earliest meaningful point], then clearlya prior time period is not possible, and the process can proceedstraight to the check at step 3728.

If the starting timepoint is something other than [time zero] or[earliest meaningful point], then the user is asked whether the factmight have been true before the starting timepoint (step 3722). Theuser's response is tested (step 3724), and if he/she is confident thatthe fact is not true, the process creates a prior [timeperiod] objectfrom [time zero] to the starting timepoint (step 3726) which can be usedto assert the inverse of the fact.

In the preferred embodiment, just as a prior [timeperiod] object mightbe created, so a check is made to see whether the second timepoint is[iafter] (step 3728), and if it is not, a [timeperiod] object for theperiod after the fact ceased to be true—from the second timepoint to[after]—is created (step 3730). Finally the [timeperiod] object iscreated (step 3732) and the process terminates.

5.10.13 Source Selection

In the preferred embodiment all assessments are associated with asource. The same method for obtaining the source information is used bythe “add_object”, “add_relation”, “add_class”, and “add_fact” processes.This method is illustrated in FIG. 38.

The behaviour will depend on whether or not the user has supplied a URLas the source, so the user's input is tested initially (step 3802). Ifthe source specified by the user is not a URL, the “select_object”process is initiated in order to identify, or, if necessary, add thesource as an object (step 3804—described in section 5.10.6). A check isthen made on the source (step 3806) to establish whether it is animate(a person or an organisation) or inanimate (e.g. a book).

If the source is inanimate, then an attempt is made to find an animatesource behind the specified source (if, for example, the source is asingle-author book, then this animate source would be the author). Acheck is made to see whether the knowledge base already knows theanimate source associated with the source specified by the user (step3808).

If the source specified by the user is a URL, the user is shown thecontent of the page at that URL and asked to confirm that that page isthe intended source document (step 3810). If it is, a copy of thecontent is stored locally and associated with an ID, unless the samepage is already held (step 3812), and a check is made to see whether ananimate creator source—in most cases this will be the site'swebmaster—is already known for the document (step 3808).

If the animate source is not already known, the user is asked whetherhe/she knows of an animate source, and, if so, whether this animatesource is responsible for all knowledge obtained from the specifiedsource or just this particular piece of knowledge (step 3814). Theuser's response is tested (step 3816). If the user does know of ananimate source, the “select_object” process is initiated in order forthe user to specify that animate source (step 3818—described in section5.10.6). If the user has said that the animate source is responsible forall information in the original source, then this fact should beasserted (step 3820), so that steps 3814 to 3820 can be omitted byfuture users who give the same source.

Finally a source ID is returned: of the animate source if one has beenestablished (step 3822), or, failing that, of the inanimate source (step3824).

5.10.13.1 Section of Document Enhancement

When a user gives a URL or otherwise indicates a document as the source,some embodiments can prompt the user to additionally highlight the partof the document that is relevant for the fact at this stage and storesufficient information to identify this section (e.g. the position ofthe start and end characters of the highlighted section).

One advantage of this is that the part of the document could behighlighted in output produced by the front end. When a fact is used toanswer a question, the section of a document from which this fact issourced can also be displayed or highlighted in a document list. Ashorter source of a fact can also enable faster verification by anyonereviewing the fact.

A change to the section of the document that has been highlighted canalso be interpreted as possibly implying that the fact has changed fortransient facts. For example, a corporate website listing the company'scontact details might update the sentence with their phone number iftheir phone number changed. This change could be brought to theattention of a user or staff member and the fact updated accordingly. Insome embodiments the change in the knowledge base would happenautomatically.

An attribute applied to a website which implies that it is keptup-to-date ([currently maintained site] perhaps) can be profitably usedin embodiments which do this by asserting such facts about the site (orrecording the information in an alternative form). The attribute meansthat incorrect facts are likely to be corrected so if the section of thedocument containing the fact is seen to be the same on a site with thisattribute, the system can infer that the fact is likely to still betrue. Examples of currently maintained websites might be the officialcorporate site of a business with staff keeping information up to date,or sites like Wikipedia which have a substantial user base correctinginformation.

5.10.14 Adding Generators

In the preferred embodiment dumb generators can be added to the systemvia a web-based editing page allowing the generator to be added to thelist and tested by the user.

The user who has created the generator is associated with the generatorand prior to editor approval the generator will be ignored by the queryanswering system for all users other than the user who has submitted thegenerator. In this way, any mistakes or errors with the generator willonly affect the user who is testing it.

After editor approval the generator will be used by the query answeringsystem for all queries.

For smart generators the system also needs to permit the addition andtesting of tools.

The preferred embodiment achieves this by allowing users to add tools inan interpreted language which can run on the server but without havingaccess to any sensitive files. The server would also terminate anyscript that was running for anything other than short timeout to preventscripts which loop. Access to the network is also controlled.

The language Python (http://www.python.org/) is a suitable scriptinglanguage used by the preferred embodiment. The interpreter is widelyavailable, freely licensed and information about how to incorporate itinto a server is widely available.

The Python script that implements the tool can again be edited andtested by the user prior to approval by an editor. On approval the toolis then available to be used in generators. Prior to approval it willonly be used in queries run by the user who submitted the tool so thatit can be tested.

5.10.15 Adding Profile Templates

The ability for users to add profile templates works in a similarfashion to generators though as there is less security risk, variousembodiments will optionally allow unapproved profiles to be seen andused by users. On approval the profile will become part of the systemand used automatically in the case where it is the only profile for themost specific class of an object.

Creation of the profile can be achieved by a web-based editor or thetemplate can be created offline and uploaded to the system.

5.10.16 Adding Translation Templates

The addition of translation templates can be achieved in a very similarmanner to adding dumb generators.

The templates, pattern and generators, are added via a web-based editorand initially only used in response to translations by the user whoadded it to allow testing.

Once the user is happy, a web-based command allows the user to submitthe template for editor approval. On editor approval the translationtemplate is used for all translations by the complete user baseextending the functionality of the system for everyone.

Various embodiments can draw attention to existing translation templatesand thus educate users in adding them by producing an explanation of howquestions were translated when a translation is successful (containingat least a link to the template used to do the translation). Thefall-back strategy when a question was not understood can also provide alink to the add_translation process with instructions thus providing theuser with a mechanism to correct and improve the problem for all users.

5.10.17 User Assessment

As used herein “user assessment” is the facility for users of anembodiment of the invention to provide information on the veracity ofknowledge already present in the system. User assessment is an optionalbut desirable feature of various embodiments as it enables users to drawattention to facts which are incorrect and/or to increase the confidencein facts which are true.

In the preferred embodiment, users can both endorse and contradictfacts. When doing so they use the same source of knowledge methodologyas is used when asserting new facts. (See section 5.10.13.)

When a user adds a fact that is already in the static knowledge base,the preferred embodiment simply considers this a user endorsement of thefact and doesn't create a new fact in the static knowledge base. Theinitial assertion of the fact also counts as an endorsement of the factby the asserting user.

The preferred embodiment also enables users who are contradicting a factto label the original fact as probably asserted abusively. Bydistinguishing between facts which were asserted in good faith but arewrong in error and facts which were probably asserted to be mischievousand/or abusive, a number of options become available. These includetaking sanctions against the user entity reporting the fact abusively,having a lower threshold for suppression of other facts asserted by thisuser and suppressing the abusively asserted fact faster than wouldotherwise have been the case.

The preferred embodiment also enables users of sufficiently high rank tolabel their assessment as final. Once done, the status of the fact (trueor false) is locked down and cannot be changed by further assessmentsfrom users of lower rank. This facility enables a highly ranked usersuch as a staff member to resolve an issue with a fact immediately. Forexample, a staff member can make an obviously abusively asserted factimmediately invisible.

In the preferred embodiment user assessment is implemented bymaintaining an assessments database table which records each endorsementand contradiction and includes the following information: the fact beinguser assessed; whether it is an endorsement or contradiction; the dateand time of the action; the reporter (i.e. the id of the user who isperforming the assessment); the source of the information (which mayalso be the user); optionally the id of the document which thisassessment is based on (if there is one). (If a document is present, thesource is the entity responsible for the document); whether theassessment has been labelled as abusive; whether the assessment has beenlabelled as final; any text explanation entered by the user at the sametime (this can be used to explain the assessment further if the userwishes and appears on the fact profile).

The user assessments of a fact are combined together to get an overallpicture of the veracity of the fact. In various embodiments, once athreshold has been reached the fact is also closed for further userassessment. This gives some stability to the system as facts for whichthere is an overwhelming certainty of them being true or false cannot bechanged. This is especially important for certain core facts usedfrequently by the system in numerous situations such as properties ofcommon relations.

Should a fact be locked down in an incorrect state, various embodimentswould however, allow a user to draw this issue to the attention of stafffor correction.

In one embodiment, user assessment information is combined together byattaching a positive score to each endorsement of a fact and a negativescore to each contradiction and setting the truth and visibility of thefact based on the sum. The magnitude of the score for each endorsementand contradiction is determined by the track record of the user makingthe assessment. For example, a new user could be given a score of 10while an experienced user who had been using the system for many monthswith a track record of accurate assessment could be given a score of200. This embodiment does not allow repeated endorsements by the sameuser to increase the sum but users can be permitted to change theirendorsement by contradicting a fact they have previously endorsed etc.

In the preferred embodiment, each fact is labelled as true/false andvisible/invisible in the table in which they are stored. true/false isthe veracity of the fact: whether the system believes it is true.visible/invisible is whether the fact is normally visible to the queryanswering system. untrue facts are always invisible. Other embodimentscould remove untrue facts from the knowledge base.

Various embodiments also take into account fact exposure information inthe assessment of the veracity of the fact from user assessments. Factexposure information is information about the events when the fact wasshown to users and the users were given an opportunity to apply a userassessment. For example, if a user has asked a question and the summaryexplanation has been displayed showing the fact and giving the user achance to contradict it, that would be an exposure of the fact to theuser. By combining exposure information with user assessments the systemcan obtain a superior understanding of the likely veracity of the fact.For example, a fact which has been exposed one thousand times andreceived five user contradictions is more likely to be true than asimilar fact which has also received five contradictions but has beenexposed far fewer times.

One example embodiment of how the system can incorporate fact exposureinformation into a scheme for assessing the fact is to consider eachexposure of a fact without a user assessment action as a form of tacitendorsement of the fact and to count these in a similar way to actualendorsements by the user but with a much smaller weight.

Various embodiments have special handling for true-now assertionsincluding taking into account the date that the endorsement orcontradiction took place. The assertion of the negative version of thetrue-now fact suggests a point when the fact may have ceased to be true.By hypothesising each of these candidate points and summing subsequentuser assessment data a similar technique to that described above can beused to assess true-now facts and determine that they should besuppressed or removed. In various embodiments true-now facts are thusalways challengeable. When the true-now fact is a temporal partnerclosed with the [iafter] object the closing timepoint of overlappingsimilar facts provides candidate change points.

5.10.18 Process for Endorsing/Contradicting Knowledge

FIG. 39 illustrates the preferred embodiment user assessment process ofendorsing or contradicting a fact in the knowledge base. The process isalways initiated with parameters for the fact to be assessed and thetype of assessment (endorsement or contradiction). First, a check ismade as to whether assessment of the fact is allowed (step 3901).Certain facts are marked as being unchallengeable, while others aresuppressed for knowledge addition. If assessment is not possible theprocess terminates and the user is given an explanation.

Then it is necessary to check whether the user is currently logged in(step 3902), and if not, he/she is required to log in (step 3904—the“authenticate” process illustrated in FIG. 19 and described in section5.9).

Next (step 3906) the system determines whether the fact is transient (oris itself a temporal partner to a transient fact). If it is not, theuser can be taken directly to the step where a source is requested(3922). If the fact is transient (or a temporal partner), it will benecessary to show the user all the other facts associated with the fact,and find out exactly what it is that the user wishes to endorse orcontradict (for example, if a user follows a link to contradict the factthat two people are married, it is not clear whether he/she iscontradicting the fact that they are married now, or the fact that theyhave ever been married). The basic “subject” fact associated with thefact being assessed is found, and a “time history” for that fact isconstructed (step 3908), indicating periods when it is true, when it isfalse, and when its veracity is unknown. The user is shown a schematicrepresentation of this time history (step 3910), and given variousoptions (to endorse or contradict particular periods, to contradict thebasic fact in its entirety, or to make changes to the time history). Ifthe user has chosen to contradict the basic fact (step 3912)—forexample, saying that two people were never married, rather than just notmarried now—then he/she is taken straight to step 3922 (specifying asource). If the user is not contradicting the basic fact, the processcontinues by checking whether the user has asked to change any of thetimepoints associated with the fact (step 3914). If there are notimepoints to change, the process checks that the user has endorsed orcontradicted at least one of the periods (step 3916), and if not, he/sheis taken back to the page at step 3910 with a message requesting atleast one endorsement, contradiction, or alteration. If there aretimepoints to change, the user is asked for them one by one (step 3918),and they are checked for validity. Next (step 3920), a new “timehistory” is constructed, based on what the user has said. If there areany inconsistencies (a fact appearing to be true and false at some pointin time, for example) the user is asked to resolve them. (Someembodiments will now show a representation of this new time history tothe user for confirmation. If the user rejects the new representation,he/she is taken back to step 3910.)

The user is then asked for the source of his/her knowledge about thefact(s) (step 3922), and a check is made on whether that source is theuser him/herself or a secondary source (step 3924). If the user is thesource, then he/she will be recorded as the animate source behindwhatever assessments and assertions are made (step 3926). If the user isnot the source of the fact, then the specified source must be identifiedand an attempt made to establish an associated animate source (step3928—illustrated in detail in FIG. 38 and described in section 5.10.13).

Finally (step 3930), the information given by the user is examined, andall assessments and assertions that follow from it (whether directly orby inference) are made.

5.11 System Assessment

As used herein “system assessment” is the automated analysis of a factto determine its veracity using at least whether the fact issemantically contradicted by other knowledge in (or known to) thesystem.

The preferred embodiment also determines whether a fact is superfluous:i.e. whether it can be generated by the system anyway.

As used herein “interactivity information” is data about how the factinteracts semantically with other facts in the system: whether a fact iscontradicted or rendered superfluous by other facts in the knowledgebase. A fact which is contradicted is in semantic conflict with one ormore other believed-true facts in the system. A fact which issuperfluous can already be produced by the system. A fact which is“uninfluenced” is neither contradicted nor superfluous and thus adds tothe total knowledge of the system.

System assessment is a useful (but optional) component found in thepreferred embodiment. It helps to keep the facts in the static knowledgebase consistent with each other and is also another weapon to counterabusive or accidental assertion of untrue facts by users. Embodimentsmaking use of user assessment data but not including system assessmentwill need an automated process to combine the user assessment data indetermining the veracity of the fact (as described above). However, inthe preferred embodiment user assessment data is used in combinationwith interactivity information in assessing a fact.

To generate interactivity information for a single fact in the preferredembodiment, the system assessment component creates a truth query infull mode corresponding to the fact.

If the fact being assessed is already in the static knowledge base italso tells the query answering system to ignore it when answering thisquery. Alternatively, the fact can be temporarily suppressed or removedfrom the static knowledge base while it is being system assessed.

The query is then executed. If the result is “no”, the fact iscontradicted. If the result is “yes” the fact is superfluous. If theresult is “unknown” the fact is uninfluenced.

A variant of this is create an inverse query corresponding to thenegative of the fact. If this query returns “yes”, the fact iscontradicted. This variant may be useful in embodiments where “no”answers to truth queries are not supported. (See section 5.5)

In embodiments producing summary explanation information, the staticfacts used in answering the query together are the ones which render thefact contradicted or superfluous and are termed “influencing facts”herein.

When the system assessment is done in the course of a user interactionand user assessment is part of the embodiment, these influencing factsare displayed to the user and an opportunity can be given for them tocontradict any they disagree with. FIG. 40 shows this.

The first thing done is to scan the record of user assessments for thisfact (endorsements and contradictions) to create a weighted sum (step4002). The sum initially starts at a small positive amount, endorsementsadd to this sum and contradictions subtract from it. The amount added orsubtracted for each assessment is a pre-determined amount based on thetrack record of the user making the assessment. The initial assertion ofthe fact is considered as an endorsement. Multiple endorsements orcontradictions by the same user are ignored.

The sum is then used to set provisional values for the veracity of thefact and its challengeability (step 4004). For example, a score abovezero would set the veracity to true (i.e. the fact is believed true),and below zero to false (believed false). Challengeability is set basedon the sum being above or below a much higher threshold. e.g. a sum lessthan −1000 or greater than +1000 would make the fact unchallengeable.

Step 4006 creates a simple truth query of just the fact itself (withoutfact id) and no query variables.

In step 4008 the query is executed in full mode with explanation. Thefact itself is temporarily masked while the query is being run, e.g. bypassing the fact id to the process_query routine and asking for thestatic search routine to ignore it. (Some embodiments may perform systemassessment on a fact before it is added to the static knowledge basemaking this masking step unnecessary.)

The return result of the query is then examined (step 4010).

If the query returned “no” (i.e. the static fact is contradicted by whatwould be in the system without it), veracity is set to false (i.e. thefact is believed untrue) and the interactivity is set to “contradicted”(step 4012).

If the query returns “unknown”, the veracity is left as set by the userassessments and the interactivity is set to “uninfluenced” (step 4014).

If the query returns “yes”, the fact doesn't appear to add knowledge tothe system that it wasn't able to generate so the interactivity is setto “superfluous” and the veracity is set to true (step 4016).

If the query returned yes or no, the concise explanation will be a listof other static facts which either implied or contradicted the staticfact being assessed. The related_facts table showing this relationshipis updated with these (step 4018).

Next, step 4020 is done to record the results of this system assessmentin the static knowledge base including the values for veracity,challengeability, interactivity and visibility. The visibility is alwaysset to false if the fact is believed untrue and in some embodiments itwill be set to invisible if the fact is superfluous. The date and timewhen this system assessment was done is also recorded for use by thesystem selecting facts for periodic reassessment. Some embodiments maychoose to remove untrue facts from the knowledge base rather than justmaking them invisible.

Next a check is made to see whether the veracity has changed as a resultof this assessment (i.e. it is now believed true when it was previouslybelieved false or vice versa) (Step 4022).

If the veracity is changed a scan of the related_facts table is madefinding facts which are influenced by the one just assessed (whoseveracity has changed) and each of these facts is recursively systemassessed (step 4024). For example, if a true fact was being contradictedby the fact just reassessed and the fact is now false, this wouldresuscitate the wrongly suppressed fact immediately.

As there may be pairs of facts which influence each other, thisrecursive reassessment needs to take care to avoid creating an infiniteloop. One way this can be achieved is to keep a log of all facts thathave been reassessed and for the routine to return without furtherrecursion if the current fact is in the log.

An example of system assessment in operation is illustrated in FIG. 41

In screen 4102 a user has typed the fact “Florence Nightingale was bornin Florence” into the first stage of the add_fact process (see section5.10.10).

This natural language was correctly translated by the natural languagetranslation system (see section 5.6) and after confirmation the fact isasserted.

As part of the fact assertion process a system assessment was done onthe newly added fact. To the user's surprise, this system assessmentresulted in the new fact being listed as contradicted. This status andthe static facts used to contradict the fact (taken from the conciseexplanation of the system assessment query) are listed (screen 4104).(This embodiment has stored the newly added fact but as it iscontradicted it has been labelled as invisible to stop it from beingused in query responses. An alternative embodiment may have chosen notto accept the fact at all.)

The user examines the list and sees that one of the facts in the list iswrong. A previous user has incorrectly asserted that FlorenceNightingale was born in England. However, as insufficient numbers oftrusted users have confirmed this fact, it is possible to contradict itand a “contradict” link is available to the user (4103).

The user clicks the contradict link (4103) and is taken into the userassessment subsystem where the user asserts the fact is false. The userassessment posts a contradiction in the assessments table for the“England is the birthplace of Florence Nightingale” fact and for goodmeasure asserts the fact that “England is not the birthplace of FlorenceNightingale”. The results of this activity is shown in screen 4108.

As it happens the user's contradiction of the incorrect fact has beenenough for the system to change the veracity of the fact from true tofalse. This has automatically resulted in the originally asserted,invisible fact (“Florence is the birthplace of Florence Nightingale”)being system assessed again and as the knowledge base no longercontradicts this fact, it has been reassessed as true, made visible andis no longer contradicted.

4110 shows the results of the question “Is Florence the birthplace ofFlorence Nightingale?” which is now correctly answered yes, showing thatthe original incorrect fact is now visible.

5.11.1 Periodic Reassessment of Facts

Various embodiments will periodically re system assess each static factin the knowledge base. In the preferred embodiment, this is achieved byhaving a field in the database table containing the static facts whichgives a date and time when the fact was last system assessed. Periodicreassessment is then achieved by calculating the timepoint correspondingto a threshold time period before the current time (e.g. one week) anddoing a SQL SELECT statement which gathers the ids of all facts whichhave not been reassessed for this period ordered by last reassessmenttime (earliest first). The program then reassesses each fact in ordertiming out after a pre-determined period (e.g. twenty minutes). A cronjob is set up to periodically (e.g. every hour) call this function sofacts are continuously reassessed. Some embodiments may prioritisecertain types of fact for faster/higher priority reassessment.

This periodic reassessment of facts ensures that things are kept up todate.

5.11.2 Periodic Reassessment of Reporters and Sources

As users are continuously adding facts and citing sources and as auser's track record is useful in various embodiments for assessingconfidence or promoting/demoting the user to various ranks, reportersand sources get periodically reassessed in the preferred embodiment.

This is achieved in a similar manner to the periodic reassessment offacts.

5.11.3 User Initiated System Assessment

In the preferred embodiment users can additionally reassess a fact atany time. This is accomplished by a “reassess this fact's properties”button on the fact profile (an example is 1409 on FIG. 14). Clickingthis button immediately results in a system assessment being done on thefact and the results displayed to the user.

5.11.4 More Specific Dates/Timeperiods

An example of how system assessment is useful is with the accuracy ofspecified dates.

Sometimes new facts added are not in semantic conflict with facts thatalready exist in the knowledge base but rather are more accurateversions of them.

An example would be a wider timeperiod associated with a fact than wasknown previously (e.g. an earlier starting time) or a more specific dateof birth for a person.

With knowledge generation and system assessment this is readily achievedby the use of generators which generate the less specific fact from themore specific, e.g. a temporal partner assertion that a fact is true forthe timeperiod [timeperiod: [timepoint: [“1930”]]; [iafter]] cangenerate the same fact with the timeperiod [timeperiod: [timepoint[“1990”]]; [iafter]]. This means that if someone were to initiallyassert the 1990-timeperiod and someone were to later assert the 1930timeperiod, the earlier fact would become superfluous and invisible. Italso means that overlapping timeperiods for opposite facts can result infacts being contradicted.

Similarly if someone were to assert a year as a date of birth and thensomeone were to later come along and provide the precise calendar date,the original less specific fact would become superfluous leaving themore specific one published.

The generators for these examples are here:

The following generator accesses the tool that calculates whether atimepoint or timeperiod is a more accurate version of another one andgenerates the fact if it is.

[tool.ismoreaccurate1@semscript.com] generator => ismoreaccurate1@locala$ [is a more accurate version of] b$ *

This generator would for example, generate a fact asserting a year as abirthdate from a fact asserting a date in that year.

[generator.ismoreaccurate2@semscript.com] generator r$ ~[equals] [is amore accurate version of] [timepoint: ts2] r$ b$ [timepoint: ts2] [is amore accurate version of] [timepoint: ts$] => [timepoint: ts$] r$ b$ *

Note that in the target line both dates need specifying. This generatorhas the effect of rendering less accurate dates superfluous when newmore accurate dates are specified. If the new more accurate date were toever become considered false (e.g. after user assessment) the lessaccurate date would then become visible again when reassessed. A similargenerator does the same for the right object.

This generator implements the concept of timeperiods overlapping;

[tool.overlap1@semscript.com] generator ! one timeperiod overlappingwith another => overlap1@local a$ [is overlapping with] b$ *(a similar generator implements the negative relation)

This generator implements the concept of one timeperiod falling withinanother:

[tool.containedwithin1@semscript.com] ! one timeperiod contained withinanother. generator => containedwithin1@local a$ [is contained within] b$*(and again an almost identical one implements the negative relation)

This generator makes the smaller timeperiod for a temporal partnersuperfluous:

[generator.tperiodimplies1@semscript.com] generator f$ [applies fortimeperiod] tp1 tp$ [is contained within] tp1 => f$ [applies fortimeperiod] tp$ *

And this one generates contradictions to facts where the timeperiodoverlaps the negative version of their subject fact:

[generator.tperiodimplies2@semscript.com] generator f$: a r b f2: a ~r bf2 [applies for timeperiod] tp1 tp1 [is overlapping with] tp$ => f$~[applies for timeperiod] tp$ *5.11.5 Assessment of Veracity (Alternative Embodiment)

As discussed in section 5.11 above, the preferred embodiment assessesveracity for uninfluenced facts by summing up a score based on eachendorsement and contradiction of the fact with the original assertion ofthe fact counting as an endorsement. Endorsements add a positive score,contradictions a negative score. If below a threshold the fact isconsidered false and is no longer used in query answering, otherwise itis considered true. If above a high threshold the fact is considereddefinitely true and user assessments are no longer accepted. The weightof each endorsement or contradiction is assigned by the track record ofthe user—users with a long track record of providing good knowledgehaving a high contribution.

The preferred embodiment only uses the reporter information, partly forsimplicity and partly because frequently users have a choice of sourcesfor a fact and are able to at least partially assess the reliability ofa source themselves. By always penalising the reporter even if it is thesource that is wrong, the reporter has an incentive to find reliablesources.

However, an alternative embodiment can attempt to rate reporters andsources by their track record and take an approach based on thecalculated probability of a fact being true using this information. Thisis an alternative philosophy where a user is largely inoculated againstthe negative consequences of accurately citing inaccurate sources.

An embodiment using this approach is now described:

The method used to calculate confidence for a fact is designed toapproximate and communicate the probability that the fact asserted istrue. Every entity is given a probability that any fact asserted by itis true. This probability is estimated initially from experience for anew entity and then adjusted as that entity develops a track record offacts and a more accurate figure can be calculated. In variousembodiments the initial probability may also be also be estimated fromthe class the entity is in. For example, a member of the class [tabloidnewspaper] may have a lower initial confidence score than a publicationin the [broadsheet newspaper] class. In other embodiments an editor mayadjust or set the initial probability based on his/her personalassessment of the reliability of the source.

5.11.5.1 Single Direct Source

In the case of a fact being asserted directly by a single user entity,the confidence figure for the fact being true is determined with theformulap _(f1) =p _(u1)where p_(f1) is an estimate of the probability that fact f1 is true andp_(u1) is an estimate of the probability of a fact asserted by user u1being true. p_(u1) is initially estimated from experience with other newusers and then modified up or down as the user establishes a trackrecord of asserting facts whose veracity or otherwise is laterestablished. One method is to look at the number of past facts that havebeen shown to be true (T_(u)) for a particular user, the number thathave been shown to be false (F_(u)) and use the ratio of the two tocalculate p_(u) with this formula:p _(u)=(T _(u)+(r _(u) *b))/(T _(u) +F _(u) +b)where r_(u) is the initial assessment of the probability and b is athreshold number used to buffer the calculation from big changes inprobability when the track record is small, e.g. b could equal 100.

This formula means that the initial estimate dominates until the pointwhere the user establishes a track record, after which the initialestimate becomes less and less important.

For example, supposing r_(u) was estimated at 0.6, b is set to 100, andthe user has asserted 9 true facts and one that turned out to be false(the user is still relatively new to the system), so T_(u) is 9 andF_(u) is 1.

In this case p_(u) will work out at (9+60)/(9+1+100) or 0.627, onlyslightly better than the initial estimate.

However, supposing the user continues his/her/its track record ofbeating the initial estimate and sometime later has 927 true facts and 3false.

In this case p_(u) will work out at (927+60)/(927+3+100) or 0.958, i.e.the user's track record has greatly improved the confidence the systemnow has in its assertions, and the initial estimate now only has a smalleffect.

5.11.5.2 Multiple Direct Sources

To assess the confidence for a fact when other users have asserted thesame fact is done by estimating the probability of the fact being true.This is one minus the probability that it is false. If severalindependent sources assert a fact, the probability the fact is true is:1−(1−p ₁)(1−p ₂)(1−p ₃) . . . (1−p _(n))Where p_(k) is an estimate of the probability that a fact asserted byentity k is true.5.11.5.3 Indirect Sources

When an indirect fact is asserted, the probability that the fact is truewhen looked at by itself is p_(u)*p_(e), i.e. to be true it has to bethe case that the user is accurately asserting that entity e asserts thefact, and that entity e is asserting the fact correctly. The confidencefigure for entity e is calculated using a similar method to that for auser entity.

Where user entities assert that another entity asserted a fact, theultimate entity asserting the fact is considered for the purposes ofcalculating confidence in the same way as a user above. The differenceis that the one or more users asserting this entity as the source of thefact are first assessed to estimate a probability that the assertion wasactually made, and this is incorporated into the calculation.

To put it another way, the probability of two facts is calculated.First, that the entity did assert the fact (in other words, the magazinedid actually say what the user says it did) and secondly that the factasserted by the magazine (say) is true. The confidence of each of thesebeing true is assessed using similar methods to the case when the factsare asserted directly by the user. These two confidences are combinedusing normal probability calculations and the methods described herein.

5.11.5.4 Independence of Sources

An enhanced version of the formula in some embodiments also considersthe possibility that the sources are not independent. For example, thedifferent users are collaborating for fraudulent reasons, or haveincorrectly asserted the knowledge as coming directly from themselveswhen in reality there is a single source. An indirect version of thiswould occur when two newspapers both publish the same story and onenewspaper is using the other as its only unacknowledged source.

In this situation an estimate of whether the users (or indirectentities) are independent is first used, estimated from experiencesimilarly to the way described above. This probability is i. Theprobability of the users all being the same source is first calculated.In the preferred embodiment this is done by calculating the mean of allthe probabilities which would have been the case if each user hadasserted the fact in isolation. Alternative embodiments take the highestor lowest probability and use that. If a is the probability assuming allsources are independent and b is the probability assuming all sourcesare the same, an overall confidence figure can be calculated using theformula: ia+(1−i)b.

5.11.5.5 Assessing Veracity

Assessing whether a fact is true or false for the purposes ofcalculating a user confidence score can be done by looking solely atfacts approved or rejected by an editor, or at facts that have beenverified by other users subsequently asserting identical facts whichtogether resulted in the assertion being considered true. In embodimentswhere a confidence figure is involved the count of true and false factscan be derived from summing their probabilities, e.g. 10 facts each ofwhich has been calculated at 90% likely to be true from other sourceswould count as 9 true facts and 1 false fact.

Because user entities are rated partly by their track record ofasserting true and false facts, some embodiments also provide anincentive for a user to both assert knowledge and to take great carethat the facts asserted are accurate.

5.12 Pregeneration of Static Facts

In section 5.4 various methods were described to generate neededknowledge primarily in response to the execution of a query.

Some embodiments of the present invention additionally or instead,generate facts prior to the execution of a query and store these factsin the knowledge base.

Pregeneration enables faster responses to queries in some circumstancesby preventing the need for the fact to be generated as part of the queryprocessing. Additionally, in embodiments where the facts are stored in arelational database, it may enable some queries to be answered using ajoin of two or more lines where all the necessary facts have beenpregenerated.

In the preferred embodiment, pregeneration is implemented as follows.

5.12.1 Pregeneration Rules

Pregeneration rules embody which facts are pregenerated and in whatcircumstances.

These are like generators but are used to specify the creation ofsuperfluous static facts as part of the pregeneration process ratherthan facts which are generated dynamically as part of query processing.Pregenerators comprise at least a trigger line, which is a pattern to bematched against static facts, a header query which tests and providesvalues and a footer which contains one more facts which can bepregenerated.

An example pregenerator is as follows:

trigger: f: x [is an occupation of] y header: query tp f [applies fortimeperiod] tp => f2: x [is an instance of] y f2 [applies fortimeperiod] tp

This example generates static [is an instance of] facts to sit alongsidefacts where an occupation is asserted. It could be used in embodimentswhere occupations are also subclasses of [human being]

The trigger line is part of the pregenerator that specifies when therule should be used. In this case it is the existence of a static factwith [is an occupation of] as the relation.

The variables in the trigger line are substituted for the entitiesactually present in the static fact and the header query is then done.So if the static fact was:

[fact.2312348@semscript.com]: [singer] [is an occupation of] [jenniferlopez]

The header query would be

query tp [fact.2312348@semscript.com] [applies for timeperiod] tp

This would return a value for tp of the 1^(St) of June 1999 onwards sothe pre-generated facts would then be:

f2: [jennifer lopez] [is an instance of] [singer] f2 [applies fortimeperiod] [timeperiod: [timepoint: [“1999/6/1”]]; [iafter]]which would go into the static knowledge base (with a suitable IDsubstituted for f2).

Another example of a pregenerator is the following:

trigger: f: x [is geographically located within] y header: query z,tp f[applies for timeperiod] tp1 f: y [is geographically located within] z f[applies for timeperiod] tp2 tp [is the timeperiod intersection of][group: tp1; tp2] => f2: x [is geographically located within] z f2[applies for timeperiod] tpwhich pregenerates all the [is geographically located within] facts foran object with larger geographical areas enabling the system to respondmore swiftly to queries where this knowledge is needed than if this hadto be inferred during query processing.5.12.2 Implemention of Pregeneration

Pregenerated static facts are labelled as such via a database flag andare treated differently from other static facts.

In the preferred embodiment pregeneration is an additional function ofsystem assessment.

System assessment without pregeneration is described in 5.11. Inembodiments which contain pregeneration it does additional steps tocreate pregenerated facts and manage them.

The first additional step is that after system assessing a fact asbelieved true, it is matched against all trigger lines of all thepregeneration rules and the resulting pregenerated facts (where theydon't exist already) are added to the knowledge base, labelled aspregenerated. In the preferred embodiment this step is only done whensystem assessing static facts which are not pregenerated. However,embodiments where pregenerated facts can trigger pregeneration rules arealso feasible. When pregenerated facts are created, the related_factstable should be updated to show what facts the pregenerated fact isinferred from. This can be achieved by simply system assessing eachpregenerated fact immediately after creating it. In alternativeembodiments, the fact that matched the trigger plus any static factsused to solve the header query can be used.

Secondly, when a pregenerated fact is system assessed as anything otherthan true and superfluous it is erased from the knowledge base.Pregenerated facts are by definition true and superfluous and thisimplementation relies on ensuring pregeneration rules match dynamicinference rules (generators) even if those rules, for efficiency, areonly used during system assessment (system assessment only generators).In embodiments where pregenerated facts can trigger the creation ofother pregenerated facts, the erasure of a pregenerated fact shouldimmediately result in all facts that were generated by that pregeneratedfact being immediately system assessed (removing them if necessary).

When a static fact on which one or more pregenerated facts relieschanges state from believed true to believed false, the existing systemassessment process of immediately system assessing facts that therelated_facts table says are generated from this fact will thus resultin these facts being immediately removed if they can no longer besupported.

A method for creating pregenerated facts after system assessment hasassessed a non pregenerated fact as believed true is shown in FIG. 47

The first step is to match the static fact against the trigger lines ofthe pregenerators and create a list (4702). This list is then loopedthrough.

At the start of the loop, the next pregenerator header query is preparedby substituting in the values created by matching the static fact to thevariables in the trigger line. This query is then executed. (4704)

A test is done to see if the query executed successfully (4706). If itdidn't the loop continues ending if all the pregenerators have beentried (4708) otherwise any results of the query and any other variablesubstitutions from matching the static fact with the trigger line aresubstituted in to the footer facts to create the pregenerated factswhich are then added (if not already present) to the knowledge base(4710).

Next the related_facts table is updated so that the static facts onwhich the pregenerated facts rely are recorded (4712). This can beachieved by system assessing the pregenerated facts.

5.12.3 Concise Explanation

Pregenerated facts and the facts which are used to generate them arestored within the related_facts table like any other fact.

As pregenerated facts may be less useful to display to the user, variousembodiments including the preferred embodiment will substitute anypregenerated fact that might otherwise have occurred in the conciseexplanation with the non pregenerated static facts recorded in therelated_facts table as being what can infer this knowledge. Inembodiments where pregenerated facts can be used to infer otherpregenerated facts, this process may need to be repeated a number oftimes until only non pregenerated facts appear.

In the preferred embodiment, the detailed explanation does still containthe pregenerated facts in the reasoning path but the fact profile of apregenerated fact will show the facts that infer it so that a user canexplore how the inference came about.

5.13 Vertical Support

Although embodiments of the present invention are fundamentally opendomain and thus able to answer questions on any topic, the same systemsand methods can be used for question answering in one or more verticals.As used herein, the term “vertical” refers to a defined knowledge domainwithin the entirety of human knowledge. Some examples are providedbelow.

Various advantages exist for supporting verticals including the abilityto answer a higher percentage of questions than might be possible foropen domain questions by comprehensively fleshing out a vertical interms of factual knowledge. Additionally, by exposing a particularvertical though a website (say) a community of people who share aninterest in the vertical can be developed which can result in strongercommunity dynamics and a larger quantity of user generated knowledge inthat vertical area. Additionally third-party users of the system mayonly be interested in a vertical and the system knowing this can helpwith disambiguation and understanding the user's intent.

The simplest method for doing this would simply be to only populate theknowledge base with knowledge belonging to a particular vertical.However, various embodiments use methods described herein forautomatically identifying one or more vertical areas within a widerknowledge base.

5.13.1 Specifying and Identifying a Vertical

A vertical can be specified by associating with it entities which belongto that vertical.

For example, the “movie” vertical could include (amongst others) theclass [movie], the class [hollywood actor] and the relations [is thefilm director of] and [starred in the movie].

This association can be done with facts in the knowledge base (e.g.[movie] [is a member of the vertical] [vertical: [“movie”]]) or by anynon knowledge base method such a separate database table.

For classes this would identify both the class and all members of theclass as belonging to the vertical. For relations it would identify boththe relation and all facts using this relation as belonging to thevertical and for other entities it would identify them as belonging tothe vertical.

A fact belongs to a named vertical if any entities within it belong tothe vertical and a query additionally relates to the vertical if anynamed entity in it belongs to the vertical.

Embodiments which provide vertical support can answer questions throughthe lens of a named vertical by analysing the queries resulting from thetranslation of a user's question to see which (if any) of thetranslations are in the vertical. Consequently when asking about “JamesDean” in the context of the movie vertical, the only interpretation thatwould survive the disambiguation process would be an interpretationabout the Hollywood actor James Dean: all interpretations involvingother people called “James Dean” would be ignored.

This process is illustrated in FIG. 45.

The first step (4502) is to receive the natural language question andthe vertical. The specified vertical could come as a parameter specifiedin an API call or it could be implicit (e.g. by the question coming viaa website known to be in the named vertical).

Next the question is translated as described in section 5.6 resulting inone or more queries which are possible translations of the question(4504). This process might include discarding interpretations which areunlikely to have been intended as the answer (see section 5.6.7).

The various queries are then analysed to see whether they belong to thevertical by seeing if they contain any named entities which belong tothe vertical (4506).

A test is then done to see whether any of the list contains aninterpretation that belongs to the vertical (4508). If the answer isyes, all the interpretations that don't belong to the vertical areremoved (4510) and the remaining one or more queries are returned(4512). If there is still more than one interpretation remaining thesecan be dealt with by prompting the user to disambiguate or by combiningmultiple answers using the methods described herein.

If there are no interpretations belonging to the vertical, they are allreturned (so asking “who is the president of france” in the movievertical would still get an answer). Alternative embodiments couldalternatively decide not to answer questions outside the vertical andsimply return an error or nothing in this situation.

5.13.2 User Creation of their Own Verticals

In various embodiments supporting verticals users can specify andautomatically create their own vertical definitions by naming a verticaland identifying the entities (including possibly classes, relations andother entities) that belong to it.

In some web based embodiments this could result in the automaticcreation (perhaps after approval) of a specific site for that vertical(perhaps accessible via a subdomain of the main site, e.g.movies.trueknowledge.com) where all questions asked were interpreted inthe context of that vertical (using the methods described above) andwhere knowledge addition was similarly interpreted by disambiguating tofacts and entities that belong to the vertical. These vertical sitescould also filter any statistics or tools they calculate or displaythrough the vertical. For example, a league table of top contributingusers would only show facts that belong to the vertical associated withthe site and the any recent knowledge addition reported would be factsthat have entities belonging to the vertical.

This method is illustrated in FIG. 46.

The user is first prompted for the name of the vertical that they wishto establish (4602). They are then prompted to specify the vertical, forexample, by being prompted for the names of entities which belong to thevertical (4604).

The system would then create a subdomain of the main website for thevertical and record the definition of the vertical (4606) enabling thesubdomain to be a window into the main knowledge base with the knowledgeconstrained to only that which belongs to the vertical. Additionalconfiguration steps can include listing the user as the owner (or havingan alternative special status) of the new site. In alternativeembodiments the website would exist as a subdirectory of the main site.e.g. website.com/verticalname/ instead of verticalname.website.com

5.14 Preventing Abuse

Various embodiments including the preferred embodiment include one ormore abuse prevention mechanisms to prevent abusive assertions ofknowledge (new facts or user assessments).

Examples of the abusive assertion of knowledge include deliberatelyasserting incorrect facts or making false user assessments, particularlyones designed to cause distress to other users or cause other problems.

System assessment and user assessment go part of the way to addressingthese issues but other techniques have utility in certain embodiments.

A technique used in various embodiments is to label certain patterns ofknowledge as prohibited. Once facts have been identified as prohibitedthey can either be refused before they are added or suppressed from usein answering queries once they are added. In the preferred embodimentthis is achieved with facts in the knowledge base which specify suchpatterns. (Alternative embodiments could store these patterns in adifferent location.) For example:

[suppressed for knowledge addition] [applies to] [fact pattern: [objectunspecified]; [is an instance of]; [objectionable human being]][suppressed for knowledge addition] [is an attribute of scope] [factpattern].[fact pattern] is a parametered class which describes facts in terms ofpatterns of known and unknown objects. In the above example, all factsof the form:x [is an instance of] [objectionable human being] are suppressed.[objectionable human being] is a subclass of [human being] reserved forpeople who are disliked. In the preferred embodiment there are(naturally) no assertions of members—its reason for being in theontology is so additions to it can be prohibited (if the user could addthe class it would be harder to prohibit additions of members inadvance).

Denotational strings for the class include all common abusive words, sosomeone asserting that their boss is a “jerk” (say) can be understood.(To prevent the user from attaching the word “jerk” to another newclass, the pattern [fact pattern: [“jerk”]; [can denote]; [objectunspecified]] can also be suppressed along with similar translationpatterns [commonly translates as] etc.)

The preferred embodiment also provides an explanation for use when auser attempts to add a suppressed fact. This is done with facts of theform:

[“too subjective to be allowable”] [is the reason for the suppressionof] [fact pattern: [object unspecified]; [is an instance of];[objectionable human being]]

FIG. 42 shows an example of these abuse prevention measures inoperation.

In screen 4202 the user has maliciously typed “William Smith is a pig”into the system's general prompt.

The translation system has correctly translated this into two possibleinterpretations (shown on screen 4203):

The assertion that William Smith is an objectionable person (4204)

The assertion that William Smith is a farm animal (4210)

(Embodiments containing a substantial number of people would probablyneed to do some ambiguity resolution on the person as well, as the nameis fairly common. This is an example using a fictional person forillustrative reasons.)

Screen 4206 shows what happens if the user selects the first of thesechoices (link 4204)

After source selection the add_fact system matches the fact to a factpattern that is [suppressed for knowledge addition] and thus refuses toadd it to the static knowledge base at all, printing out the reason.This is shown in screen 4208. (An alternative embodiment could record itin the system but not use it for query responses.)

4212 shows what happens if the user selects the second link on screen4203 (link 4210).

Again the system asks for a source, but this time the fact doesn't matchany suppressed pattern and the fact is added to the static knowledgebase and system assessed. However, the system assessment discovers thatthe fact is contradicted and the output of this contradiction includingall the static facts which together are in semantic conflict with thenew assertion are shown (screen 4214). As all these facts areunchallengeable there isn't actually anything the user can do. Even ifthey were, they would be unlikely to change the veracity of any of themby themselves unless they were a trusted user.

The user clicks on the fact they just asserted (link 4215) and thisopens a profile of the newly added fact (4216).

Screen 4216 shows that the fact they just added is believed false,invisible and contradicted (i.e. it is not being used to answer queriesand is not displayed) so again the user has been thwarted.

Furthermore, this incorrect fact can now be used to attach a lower levelof trust to assertions and endorsement/contradictions made by this userin the future.

Various embodiments can also implement a “soft” form of this techniqueso that facts that match the pattern are allowed but are immediatelyflagged for rapid review by an editor.

Alternatively or additionally facts so matching can use deferredpublication protocol (see section 5.10.2.2) instead of immediatepublication protocol (section 5.10.2.1).

This soft suppression could be used for facts which have significantpotential to be abusive but can nevertheless be true. Examples wouldinclude asserting that a new geographical area was a US State long afterall 50 states had already been added or asserting a date of death for aprominent unpopular politician.

Another similar technique is the attribute [class is complete] which canbe applied to certain classes, e.g.

[class is complete] [applies to] [boolean]

The add_object process will refuse to allow new objects to be added toany such class (or subclass of such a class).

This prevents someone from using add_object to add (say) a third Booleanvalue or a 13th calendar month. The add_fact process also needs a checkfor facts of the form x [is an instance of] y where this attributeapplies to class y.

5.15 Knowledge Contribution Incentives

Various embodiments include one or more incentive mechanisms designed tomotivate users to add knowledge to the system. Desirably this knowledgeshould be relevant, useful and accurate.

The various methods that can be included in such incentive mechanismsare now disclosed:

5.15.1 League Table

One incentive is to publish a ranking of entities who have contributedthe most to the system. Various embodiments make this publicationoptional: users could exclude themselves from this list if they chose.Such a list can motivate some of the higher contributors by encouraginga competitive spirit and being a public display of gratitude. Being highranked could give personal kudos and may lead (in some embodiments) tobecoming an editor.

One's track record of adding facts can be displayed in a [human being]profile template focussing on statistics and contributions. An exampleis shown in screen 1414 in FIG. 14.

5.15.2 Payments

Other embodiments motivate users by promising and paying commission onearnings derived from the commercial exploitation of the knowledge thatthey contribute over a named time period.

These ideas require a measure of how much has been contributed. Oneembodiment simply counts facts added to the knowledge base by a user.Embodiments where facts are assessed as being true or false or given aconfidence score use these to calculate a count of true facts.

In the preferred embodiment ranking is done by calculating how oftenfacts contributed by a user are referenced when processing queriessubmitted by users of the system. This can be implemented simply byincrementing a count each time a particular fact is used in answering aquery. A usage count method has the virtue of encouraging users to focuson knowledge that other users are interested in looking up.

A pure count method may encourage users to add large quantities ofknowledge that nobody is interested in. Various embodiments mayadditionally publish questions/queries that users have submitted wherean answer wasn't found, in order to encourage users to contribute theknowledge missing in these cases.

Other embodiments combine both this usage count with a count of factscontributed using a weighted sum. Other embodiments further distinguishbetween routine facts used as part of the query-answering process (suchas a common relationship being symmetric) and facts that are directlypart of the knowledge the user was attempting to discover. Such criticalfacts are weighted more highly.

The preferred embodiment also has a mechanism where users can drawaspects of the published knowledge to the attention of an editor. Invarious embodiments this is accompanied either always or in certaincircumstances by the payment of a fee for consideration. In embodimentswhere both types of contact are permitted, the ones accompanied by a feeare given a high priority. The petition could be one asking for anasserted fact to be rapidly widely published.

5.15.3 Focusing Users on Needed Knowledge

When users are incentivised to add knowledge that is of maximum use toother users (e.g. the financial, commission-based incentive above)another method used in various embodiments is to log all requests whichcannot be answered from knowledge in the knowledge base. This log isalso augmented with a record of whether the requests are independent(submitted from different computers/people) and the time of the request,and enables a count to be made of how often this request has been made.If knowledge is added that subsequently allows the query to be answered,the time will show which requests should still be considered not known(ones submitted before the earliest time the query was known to beanswerable are no longer an issue). In another embodiment all identicalrequests can be updated with a Boolean flag or erased from the list whenthe system discovers a query can now be answered.

A user wishing to submit knowledge that is maximally useful can then askthe system to give them a high-frequency unfulfilled request so thatthey can research what knowledge is missing from the knowledge base inorder to add it. The system would then give the user “ownership” of thatquery for a limited time to stop multiple people competing to add thesame knowledge. Once their ownership ended, the system will run thequery internally to see whether they had succeeded and either remove itfrom the list or maintain it for another user to consider in due course.Further embodiments of the system would prioritise the most neededqueries to the most trusted (by track record) users.

In a further embodiment users could attach an amount of money to anunanswered query to financially incentivise someone else to add theknowledge. The user that got this query to add would get a percentage ofthe money when the appropriate knowledge was added. Furthermore, suchqueries would be offered first to the most trusted users in financialamount order giving users a financial incentive to build a good trackrecord: they are then nearer the front of the queue for the largerpayments. Queries where users attach money to get them answered could bequeries that they have typed in and not got a response or they could bein the form of short messages describing what knowledge they want addedto the knowledge base.

5.15.4 Usage Agreement

A further method the preferred embodiment uses to increase the accuracyof the knowledge asserted is to publish a usage agreement within thesystem, and to prompt each user to read and consent to it before usingthe system. The usage agreement contains terms that insist that onlyknowledge that the user believes to be true should be asserted, andprovides negative consequences if these terms are broken. Warningsreferring to the importance of accuracy and the user agreement are alsoincluded in the screens presenting the interaction with the system whenthe knowledge is asserted by the user.

5.16 Integrating with a Search Engine (using documents as a fall-back)

Some embodiments of the present invention work as part of, or togetherwith, an internet search engine. FIG. 43 illustrates this. The searchstring entered by the user (step 4302) is processed as a question (step4304) and the result (if any) is stored. The string is then passed on toa search engine (step 4306), possibly after refinement (as illustratedin FIG. 44, and discussed in more detail in section 5.16.1 below). Theresults from both processes (4304 and 4306) are then combined anddisplayed (step 4308).

The combination of search-engine and embodiment of the invention can beachieved in various ways.

In one embodiment, the search-engine is simply an additional componentoperable to take a query and produce an ordered list of documents. Thiscomponent can be called as a fall-back when a question is not translatedor when the question is translated but no responses are produced. i.e.The output can actually just be one result or the other.

In an alternative embodiment, the invention is implemented as a“plug-in” for a pre-existing search-engine. The primary system is astandard search-engine but the user query is additionally passed throughthis embodiment (possibly acting remotely). Any response from thisembodiment is displayed above the search-engine results or instead ofthe search-engine results. If there is no useful response, thesearch-engine results are displayed. In this way the user is no worseoff than they would have been using a standard search-engine but iftheir query can be answered using this embodiment, the user benefitsfrom the advantages of a response based on structured knowledge. FIG. 2shows an example of this.

Various embodiments handle the combination and display of results invarious ways. If the results were recognised and the knowledge baseproduced useful answers, this output could be presented either insteadof or in addition to the normal output from the search engine. Thedisplay of search engine results could depend on the user clicking alink. (In some embodiments step 4306 would be normally bypassed, andonly initiated if the user chose to click on a button or link to showsearch engine results.) If all or some of the search string wererecognised as denoting an object in the knowledge base, one or moreprofile screens for those objects could form all or part of the output.In other embodiments links to the profile screen(s) or knowledge baseresponse could be output.

The functional combination of search-engine and present inventionembodiment can be achieved in a variety of ways:

According to various embodiments, the combination of requests happens atthe server end. The user query is transmitted over the internet to theweb server which passes the request to both the search engine and thisembodiment, combines the results and presents the resulting combinedresults to the user.

According to other embodiments, the combination happens at the clientend. For example, a toolbar or web browser extension supplies the userquery to both a search engine and independently to an online embodimentof this invention, receives the results independently from each systemand acts accordingly. This action could be to display the structuredknowledge results if successful and the search-engine results if not; tocombine the two into a single page or to open two windows, one for eachset of results.

5.16.1 Keyword Refinement

In other related embodiments the knowledge base is used to enhance thequery prior to processing by the search engine. This refinement processis illustrated in FIG. 44. The query to be refined is first collected(step 4402), and then analysed and converted into a form more suitablefor keyword searching (step 4404). The converted query is then passed toa search engine (step 4406), and the resulting list of documentsreturned (step 4408). For example, when a string is recognised asdenoting an object in the knowledge base, other strings known to denotethis object can be used to form part of the keyword search, especiallywhen the present system knows that such alternative strings uniquelydenote the object being searched for (or are used for this object in themajority of cases). For example, a web search involving the string “BillClinton” could be expanded to also search for pages including the string“William Jefferson Clinton” or “President Clinton” and (optionally) totreat the alternatives equally.

A further refinement to this embodiment working in conjunction with asearch engine is some pre-processing of the query string when the queryis in question form, to make it more likely to locate the information ina web-based document—specifically removing possible keywords that arepart of the grammar of the question rather than words needed to belocated within the documents searched for. For example, if the querystarts “What is . . . ” and the answer is not located in the structuredknowledge base, the query is going to be used for keyword searching inthe hope that the search engine will turn up a document that has theanswer contained within it in natural language. As a consequence theword “what” is not a keyword that should be searched for and should beextracted from the query before passing it to the search-engine.

5.17 Remote Computer Interface

A desirable (but optional) feature present in various embodiments is toallow access by remote computers. As embodiments are able to produceknowledge in structured form, services provided, especially queryanswering, have the potential to be substantially more useful to aremote automated system than natural language documents returned by asearch engine which are designed for (and only really useful for) humaneyes.

The preferred embodiment is operable to provide several services toremote automated systems including:

Query answering. The remote automated system sends a query over thenetwork and the embodiment processes the query and sends the resultsback.

Translation. The remote system sends a natural language question overthe network and a translation into one or more queries is returned.

By providing a variety of services some components of variousembodiments (such as the knowledge addition component) can be located onremote systems on the network and use the remote computer interface tointeract with other components.

Computer communication over a network is a very established area oftechnology so a large variety of implementations will be obvious but aninteraction with the remote computer interface will typically involve:

Transmission over the network of request data specifying the requestedservice by the client automated system.

Receipt of this request data by the embodiment.

Fulfillment of the service using the request data and generating returndata (e.g. the results from processing the query).

Transmission by the embodiment of this return data over the network.

Receipt of the return data by the client automated system.

In many embodiments the request data will include authentication dataenabling the embodiment to know the identity of the remote automatedsystem and to (optionally) record and (optionally) seek considerationfor the service (for example charging money).

Other embodiments use an interactive session where a single interactionis accomplished by more than two interactive transmissions of data. Forexample, the authentication step could be done first and onceauthentication had been acknowledged the client system could then sendthe query to be answered.

The preferred embodiment is operable to receive the request data via anHTTP (or HTTPS) request where the request data is encoded using HTTPrequest variables. The embodiment responds (again using HTTP) with anXML document matching a pre-determined XML schema.

5.17.1 Example Remote Computer Interaction

Here is a sample interaction with the preferred embodiment, disclosed toillustrate the principles involved and give a concrete example of an XMLschema suitable for answering translation and query requests.

5.17.1.1 Translation Request

First the remote computer wants the question “Who are President Monroe'schildren?” translated. It sends the following HTTP request (essentiallyopening the following URL):

http://www.worldkb.can/remote_service?user=xxx1&pw=testpassword&service=question&question=Who+ are+President+Monroe %27s+ children%3F

The remote system responds with:

<semscript:response xmlns:semscript=“http://www.semscript.com/ns/remote_service” type=“question”> <semscript:status>ok</semscript:status><semscript:num_results>1</semscript:num_results> <semscript:resulttype=“query”> <semscript:query> query e|[current time] [applies to]now|f: e [is a child of] [president james monroe]|f [applies attimepoint] now </semscript:query> <semscript:retranslation> Determinewhich objects have the relationship ‘is a child of’ to President JamesMonroe, the 5th President of the United States at the current time.</semscript:retranslation> </semscript:result> </semscript:response>i.e. The query returned is:

query e [current time] [applies to] now f: e [is a child of] [presidentjames monroe] f [applies at timepoint] now5.17.1.2 Query Response

The remote system then attempts to get an answer for the query with thefollowing HTTP (or HTTPS) request:

http://www.worldkb.com/remote_service?user=xxx1&pw=testpassword&-service=query&xml=1&query=query+e%0D%0A%5Bcurrent+time%5D+%5Bapplies+to%5D+now%0D%0Af%3A+e+%5Bis+a+child+of%5D+%5Bpresident+james+monroe%5D%0D%0Af+%5Bapplies+at+timepoint%5D+now&mode=full&explanation=1and gets the following response:

<semscript:responsexmlns:semscript=“http://www.semscript.com/ns/kengine”type=“query_execution”> <semscript:status>complete</semscript:status><semscript:num_results>3</semscript:num_results> <semscript:result><semscript:variable name=“e”> <semscript:id>[eliza kortrightmonroe]</semscript:id> <semscript:unique_translation>Eliza KortrightMonroe, daughter of US President Monroe,1787-1840</semscript:unique_translation> </semscript:variable></semscript:result> <semscript:result> <semscript:variable name=“e”><semscript:id>[james spence monroe]</semscript:id><semscript:unique_translation>James Spence Monroe, 1799-1800, son of USPresident Monroe</semscript:unique_translation> </semscript:variable></semscript:result> <semscript:result> <semscript:variable name=“e”><semscript:id>[maria hester monroe]</semscript:id><semscript:unique_translation>Maria Hester Monroe, 1803-1850, daughterof US President James Monroe</semscript:unique_translation></semscript:variable> </semscript:result> <semscript:explanationtype=“short”> <a href=“profile?id=[fact.1147791%40semscript.com]”>ElizaMonroe is a child of President James Monroe</a><br /> <ahref=“profile?id=[fact.1132264%40semscript.com]”>‘is a child of’ ispermanent</a><br /> <ahref=“profile?id=[fact.1147792%40semscript.com]”>James Monroe is a childof President James Monroe</a><br /> <ahref=“profile?id=[fact.1147801%40semscript.com]”>Maria Monroe is a childof President James Monroe</a><br /> </semscript:explanation><semscript:explanation type=“long”> [...] </semscript:explanation><semscript:num_essential_facts>4</semscript:num_essential_facts><semscript:essential_fact><semscript:fact_id>fact.1147791@semscript.com</semscript:fact_id><semscript:fact_translation>Eliza Monroe is a child of President JamesMonroe</semscript:fact_translation> [...] </semscript:response>The long explanation and some other data has been removed for spacereasons but the example should illustrate the principles involved.5.17.2 Example Applications

The remote computer interface can be used any computer system wantingaccess to real world knowledge for any purpose. The number ofapplications is therefore extremely large.

However, to give some idea of the possibilities, some random exampleapplications are now disclosed which embodiments of the currentinvention may enable via the remote computer interface. This is notintended to be a definitive list but to be illustrative of what can beachieved and to further expand upon the principles disclosed:

5.17.2.1 Enhanced Email Client #1

In current email clients one must either know the recipient's emailaddress when sending an email, either directly or by having theirdetails in a local address book. Furthermore, when email addresseschange or become obsolete, the records in the local address book do notupdate.

Using the remote computer interface to an embodiment of the presentinvention containing substantial amounts of knowledge of people andtheir addresses, this would no longer be a problem. The user couldsimply type the name of the person or entity that they wish to emailinto the To: field and the email client could do a query to determinethe correct email address. For example, for “William MacDonald” typedinto the To: field, the query could be:

query urs,e [“William MacDonald”] [can denote] p p [uniquely translatesas] urs [current time] [applies to] now f: e [is the main email addressof] p f [applies at timepoint] now

Ambiguity (via unique translation) could also be overcome. In the aboveexample, the unique recognition strings of all the people to which“William MacDonald” is a valid denotational string and who have an emailaddress stored are returned. The unique recognition string or stringscan be used to confirm the intended recipient or to enable userselection in the event of ambiguity.

In an alternative embodiment it can be done in two steps, with all thepeople listed and the email address retrieved

It could also handle a miniature translation using embodiments wheretranslation is a remote service. For example, the user could type “TheCEO of Ficton Engineering Ltd” without knowing his name. The emailclient could then translate that into a query, do the query and look uptheir email address.

5.17.2.2 National Holiday Determination

Applications which need to determine whether a particular date is anational holiday in a particular jurisdiction (e.g. scheduling systems,diary applications, loan interest calculation systems) are likely (priorto access to an embodiment of the current invention) to have thisinformation stored in a local database or some form of local systemrequiring quite complicated coding.

An embodiment with this knowledge supported could enable all suchapplications to retrieve this information via the remote computerinterface without any of them needing any local coding beyond what isneeded to support access.

There are a number of ways this knowledge can be represented in variousembodiments. One exemplary method is to have a class of calendar dateswhich is a subclass of timepoint with the accuracy specified to aday/year only and to have a subclass of calendar dates for nationalholidays.

An example query could then be:

query [timepoint: [“2006/5/29”]] [is an instance of] [england and walesbank holiday]

Corresponding to the question “Is the 29th of May 2006 an English bankholiday?”

Separate classes can be added as more jurisdictions are added.

Alternatively the classes can use a parametered object with theadministrative area at issue as the parameter, e.g. for nationalholidays in the United States the class could be [national holiday:[united states of america]].

5.17.2.3 Automatic Threading of Emails by Entity

Email clients can only distinguish between different email addresses. Inpractice people have several different email addresses and over yearschange their address regularly. Without express knowledge being given tothe email client, the email client has no way of knowing that thesedifferent emails from different from-addresses came from the sameperson.

An embodiment keeping track of current and former email addresses ofpeople and entities can provide this knowledge to an email client viathe remote computer interface enabling all emails from the same entityto be grouped together regardless of the address they were sent from.

An example embodiment could implement this with the transient relation[is an email address of], left class [email address], right class[agreement making entity] (a parent class of human and organisation).The query need not specify any temporal constraints as the query isasking for all current and former email addresses of a person.

5.17.2.4 Polling for Events of Interest

By periodically doing a query an automated system can keep a userup-to-date with news about entities of interest.

For example, a program could periodically do a query to test and todiscover the marital status, life/death status, children and employmentstatus of friends, family members and acquaintances of the personwanting this service. This test could be done daily (e.g. with a cronjob) and changes reported to the user (e.g. by email).

Such a service would automatically keep a person up-to-date with eventsof great personal (but not national) interest which might otherwise onlybe communicated late or not at all and certainly would not becommunicated using the conventional media.

An automated system could use a similar technique to do certain actionsonce an event had taken place. For example, by using automated periodicpolling of a query, it would be possible to arrange to send an emailafter one's own death—perhaps sending information to loved ones that onewanted suppressed during one's lifetime.

5.17.2.5 Enhanced Word Processor Features

Embodiments of the current invention can contain a substantial amount ofnatural language data linking natural language to real-world entitiesand additionally providing much useful knowledge about those entities.

By using the remote computer interface a word processing applicationcould intelligently identify what was being described and offeradditional services tailored to what is being typed. For example, when aperson's name was being typed, an option to insert an identifyingphotograph of that person into the document could become available. Whena geographic area was mentioned, maps and driving directions could beoffered.

5.18 Advertising and Messaging Applications

5.18.1 Contextual Advertising

The knowledge systems described herein can be applied to improvecontextual advertising systems.

With current contextual advertising systems attached to search enginesor displayed on general web pages, advertisers bid on keywords. Keywordsare often ambiguous. Furthermore, there may be many keywords that arerelevant to the product or service being advertised.

In contextual advertising systems powered by embodiments of the presentinvention, the advertiser can specify the semantic concepts that theywish their adverts to be associated with (by ID) and the relevance canbe determined from facts relating natural language words and phrases tothe semantic concept(s) advertised. In some embodiments the keywords canbe generated in advance from the semantic concepts. In other embodimentstheir semantic concepts can be used to determine which adverts todisplay in real time.

In the various human question answering embodiments described herein,the disambiguation process can be used to determine more accuratelywhich adverts are appropriate.

For example, with the example “is Madonna single?” question, contextualadverts about music singles or adverts advertising businesses local toMadonna, Md. can be ignored as the disambiguation process has shown thatneither of these concepts is related to the question. Compare this witha keyword based approach where the intended sense of “single” would notbe apparent and where finding appropriate keywords for Madonna, Md.would be impossible.

An advertiser wanting to advertise a celebrity gossip magazine would beable to display their advert very appropriately against this query as ahigh degree of confidence could be established that both the celebrityand her marital status were being asked about.

5.18.2 More Intelligent Advertising

An additional way that more relevant advertising can be delivered onsystems powered by embodiments of the present invention is viaintelligent sponsorship based on knowledge generated by reference to thehuman being entity using the system.

For example, a user with a young child whose birthday was coming up inthe next few days could be presented with a highly-tailored intelligentmessage directing him to a local bakery which sells birthday cakes. Theadvertiser might be the bakery attempting to identify local parents withchildren celebrating their birthdays.

A second example, would be a drug manufacturer trying to reachindividuals who are members of a particular medical specialty.

Each of these examples can be implemented by the advertiser creating aquery based on the user id of the user which can be used to trigger orgenerate the text for the message.

e.g. For the second example:

query [current time] [applies to] now h [is the true identity of] [user:[“joesmith576”]] f: a [is the occupation of] h f [applies at timepoint]now f [is strictly equal to] [cardiologist]would return yes if the user is a cardiologist.

The first example would be similar but would generate family members andtheir birthdays for checking.

Other queries (such as could be used in the first example) couldgenerate strings to be inserted into a template message—such as thechild's name and the birthday date.

5.18.3 Intelligent Messages and Reminders

Similar to the intelligent advertising ideas above is the generation ofintelligent messages that might be useful or enjoyable to the user.

For example:

Happy Birthday

Don't forget that tomorrow is your wedding anniversary.

Today is your business partner John's birthday

The mechanism for the implementing these is similar to that describedabove with queries based on the user id (or human being id inembodiments where that is used directly) to generate the message orstrings in the message.

Supporting Documentation

6.1 Additional Materials

6.1.1 Exemplary Embodiment Ontology

The root class is [object]

6.1.1.1 Immediate Subclasses of [object]:

[conceptual object] [group] [thing with a creation date] (subclasses are[physical object] and others) [object which can have a geographicallocation] (subclasses [geographical area] [physical object] and[organisation]) [source of knowledge] [physical object] [is a distinctclass from] [conceptual object]6.1.1.2 Subclasses of [physical object]

[biological object] (subclasses [animal] [plant] [virus])[agreement-making entity] (similar to a legal person, subclasses [humanbeing] [organisation])6.1.1.3 Subclasses of [conceptual object]

[boolean] [animated visual medium] [concept with variable precision](subclasses include [geographical area] and [timepoint]) [linguisticconcept] [mass] [number] [worldkb object] (suclasses include [fact][attribute] [relation] [class] etc.) [string] [timeperiod] [timepoint]6.1.2 Additional Generators

The following is a list of generators present in the preferredembodiment (and not mentioned elsewhere).

7 High Precision Information Extraction

The following is a description of techniques for extracting structuredknowledge, e.g., facts, from unstructured text for use with knowledgerepresentation systems. Techniques for approximating the reliability ofextracted facts are also described. And while specific embodiments inthis description assume the context of a knowledge representation systemhaving features as described herein, it should be noted that theinformation extraction and reliability approximations described hereinmay be used with a broader range of knowledge representations systemsand so should not be limited by such references.

A specific embodiment of a system for extracting structured knowledgewill be described that includes two parts: a Natural Language Processing(NLP) System that extracts facts from unstructured text, and aQuarantine that establishes the reliability of the extracted facts.According to the described embodiment, facts are object triples that canbe used to form a Knowledge Base as described above. Knowledge extractedin accordance with various embodiments of the invention can enhance sucha system and be available for user question answering and other uses.

7.1 NLP System

According to various embodiments of the invention, the NLP System (asillustrated in FIG. 48) splits unstructured text (4802) into sentences,parses the sentences into parse trees, and transforms the parse treesinto short assertion statements. These short assertion statements arethen translated into facts by a Knowledge System which resides in aquarantine (4818).

The NLP System illustrated in FIG. 48 includes a Parser and aTranslator. The Parser reads unstructured text. Parses sentences andoutputs simplified assertion statements. The Translator (4816)translates assertions into facts.

An embodiment of the Parser could enrich the output with a definedMarkup. One implementation of the Markup links to unique specificentities in a Knowledge Base. These entities are disambiguated duringthe process of parsing.

The Translator takes short assertion statements, e.g., of the structure“Subject Noun Phrase, Verb Phrase, Object Noun Phrase,” and outputs facttriples of the form “Object 1, Relation, Object 2”. In a particularembodiment the Translator used is True Knowledge's Translation API,described in U.S. Patent Publication No. US-2009-0192968-A1,incorporated herein by reference above. A typical translation examplewould take “Peaches Geldof is the daughter of Bob Geldof” as input andgenerate the fact “[peaches geldof] [is the daughter of] [bob geldof]”.

In the depicted embodiment the Parser includes 4 stages: Processing,Part of Speech Tagging (POS Tagging), Sentence Parsing and SentenceTransformation. The Processor takes unstructured text as input andoutputs sentences. The Processor in addition to splitting sentences maybe configured to mark up entities occurring in sentences as describedabove. In a particular embodiment the Processor includes two parts:Sentence Splitting and Sentence Markup.

The Sentence Splitter (4804) takes unstructured text as input andgenerates a list of sentences. The Sentence Markup (4806) takes a listof sentences as input and generates a list of marked up sentences. Notall sentences given as input to the Sentence Markup will necessarily beincluded in the output. For example sentence features may be identifiedwhich are positively or negatively correlated with a sentence containingan assertion. Sentences exhibiting negative features or sentences notexhibiting positive features may be filtered. Such sentence features mayinclude, but are not limited to, sentence structure, words, punctuationand sentence length. In a particular embodiment sentence classificationis achieved through regular expressions.

The Sentence Markup may (and is not required to) mark up entities of thefollowing type: dates, currencies, quantities, named entities. NamedEntities may not only be identified and/or linked to an externalKnowledge Base but can also be classified and/or linked within thedocument being processed either within sentences or across multiplesentences (i.e., Anaphora Resolution). In a particular embodiment theGATE Information Extraction engine (open source software available viahttp://gate.ac.uk/) is used for Named Entity identification and Anaphoraresolution. An alternative embodiment could use Open Calais to performNamed Entity identification and Anaphora resolution(http://www.opencalais.com/).

In a particular embodiment True Knowledge entities (as described aboveand in U.S. Patent Publication No. US-2009-0192968-A1 incorporated byreference above) are used for mark-up. For example “Peaches Geldof wasborn on the 13th March 1989” could be marked up as “[peaches geldof] wasborn on [timepoint: [1989/3/13]]” where square brackets denote uniqueentities in the True Knowledge knowledge base.

The POS Tagger (4808), takes sentences (marked up or otherwise) asinput, and generates lists of Tokens. A token is a word, part-of-speechtuple. A list of tokens is essentially the initial sentence withparts-of-speech tags stored in parallel, e.g., “The cat sat on the mat”becomes “The/DT cat/NN sat/VB on/ON the/DT mat/NN”. The POS Taggeremployed in a particular embodiment is the Stanford POS Tagger (seehttp://nlp.stanford.edu/software/tagger.shtml) with theleft3words-wsj-0-18 language model. The POS tags used in a particularembodiment are those defined by the Penn Treebank project(http://www.cis.upenn.edu/˜treebank)

The Sentence Parser (4810) takes as input a POS-tagged sentence in theform of a list of tokens and generates a full parse tree of thesentence. A variety of parsers could be used to perform the SentenceParsing including, but not limited to, Deterministic Finite StateMachines, Non-Deterministic Finite State Machines, LL-Parsers,LR-Parsers, Probabilistic Parsers, etc. A particular embodiment uses anLR-Parser (see http://en.wikipedia.org/wiki/LR_parser). An example parsetree is shown in FIG. 49.

The Sentence Transformer (4812) takes a parsed sentence in the form of aParse Tree as input and outputs short assertion statements. In aparticular embodiment the Sentence Transformer identifies Parse Treesmatching pre-defined patterns that contain sub-trees corresponding to aSubject Noun Phrase, a Verb Phrase and an Object Noun Phrase. Thesethree sub-trees are flattened and concatenated to produce simpleassertion statements. Note that one sentence can produce multipleassertion statements. In a particular embodiment, temporal and spatialmodifiers can also be matched as sub-trees in the function describedabove. These modifiers describe where and when the fact extracted froman assertion statement occurred.

Translation rules define paths in the parse tree. These rules arematched to the parse tree and every possible combination of matchedsubject-verb-object is output as a short assertion.

Assuming that an embodiment had the following translation rules:

-   -   Sentence/compound noun phrase/noun phrase→Subject    -   Sentence/compound verb phrase/verb phrase→Verb    -   Sentence/compound verb phrase/noun phrase→Object

The example provided in FIG. 49 would be flattened into the followingsentences:

-   -   Sub-trees (illustrated in . . . )    -   Peaches Geldof lives in Islington (4902, 4906, 4908)    -   Peaches Geldof is the daughter of Bob Geldof (4902, 4910, 4912)    -   UK Socialite lives in Islington (4904, 4906, 4908)    -   UK Socialite is the daughter of Bob Geldof (4904, 4910, 4912)

During the translation phase (4816), as entities are disambiguated,these entities are propagated through later sentences (4814). Forexample, from the sentence “Vauxhall was acquired by General Motors”followed by “Vauxhall is 107 years old” and the internal document link,we know that “Vauxhall” in both sentences is referring to the sameentity. The string “Vauxhall” is ambiguous as to whether it refers tothe area in London or Vauxhall Motors. When viewed singly, the firststatement is unambiguous (because places are generally not acquired),while the second statement is ambiguous. When the Translator translates“Vauxhall was acquired by General Motors” unambiguously to “[vauxhallmotors] [was acquired by] [general motors]”, that disambiguation can bepropagated to the next sentence transforming it into the unambiguous“[vauxhall motors] is 107 years old”. An alternative embodiment, ratherthan translating sentences linearly, could translate in decreasing orderof ambiguity to maximise the volume of sentences output.

7.2 Quarantine

According to various embodiments, the Quarantine uses a system offilters to identify unreliable facts. Facts not considered unreliableare then fed into a series of buckets, each bucket representing a factcharacteristic and each fact entering multiple buckets. When full, thereliability of each bucket is statistically calculated and buckets areblack, white, or grey listed. Black-listed buckets feed back into thefiltering step, while facts residing in white-listed buckets areconsidered reliable enough to be used in a Knowledge System.

As the Knowledge System itself is used in the NLP System (intranslation) and the Quarantine (in filtering), the whole systemfeeds-back on itself and increases in scope. Any tendency for quality todegrade is significantly mitigated by the black-listing of buckets andfiltering facts about unknown objects.

The Quarantine (an example of which is shown in FIG. 50 and FIG. 51) isa system which assesses the reliability of facts. According to specificembodiments, a reliability score can be used in two ways, depending onthe Knowledge System in which the facts are destined to be used. In aprobabilistic system, facts can be assigned a single probability orprobability range and be thus inserted into the Knowledge Base. In anon-probabilistic system, facts calculated to have a high reliabilitycan be inserted into the Knowledge Base, and facts with an undeterminedprobability can be held back. Probabilities can be recalculated overtime, allowing facts in a probabilistic system to be updated, and factsin a non-probabilistic system to move to and fro between the Quarantineand the Knowledge Base.

The Quarantine takes as input a list of facts and generates areliability score for each fact. In a non-probabilistic system this maybe a Boolean, while in a probabilistic system this may be either asingle number or two numbers denoting a probability range. TheQuarantine processes each fact in turn. According to a specificembodiment, Fact Processing is performed in two stages Fact Evaluation(FIG. 50) followed by Bucket Processing (FIG. 51).

The Fact Evaluation evaluates each fact individually (5002). Initiallythe fact is compared against a series of Fact Filters, each of which canfilter the fact as unreliable (5004). If the fact is not filtered, thereare three ways it can be judged reliable:

-   -   It is contained in the relevant Required Buckets.    -   It is already believed by a Knowledge System (i.e. it is        superfluous).    -   The same fact has been extracted from sentences of different        formats from different sources (5008).

In a particular embodiment, the Knowledge System employed is the QueryAPI described herein and in U.S. Patent Publication No.US-2009-0192968-A1 incorporated by reference above.

In a particular embodiment, reliable facts are added to a Knowledge Base(5010).

In a particular embodiment, the following cases will cause a Fact Filterto filter a fact:

-   -   The fact is inconsistent with the Knowledge System.    -   The fact contradicts the Knowledge System.    -   The fact is of a sensitive or controversial nature (references        to people's sexuality, references to the existence of deities,        etc.).    -   The subject and object of the fact are the same entity.    -   One of the objects in the fact is not already in the Knowledge        Base.    -   An existential object is referred to in the sentence from which        the fact is extracted.    -   The fact is contained in a Black-Listed Bucket.    -   The fact refers to a derivatively-named object (i.e. an object        that has been named after something, hence is easily confused        with the original object).    -   The components used to parse the sentence that generated this        fact (described in the description of the NLP System) have since        been suppressed.    -   If one of the objects is a person, he/she should not be referred        to solely by either the first or second name.

According to an alternative embodiment, rather than filtering factsabout unknown objects, facts for such objects are created.

After the fact has been filtered, it is added to multiple Fact Buckets(5006). A fact bucket is defined by an individual fact characteristic.All the facts within a fact bucket exhibit this same characteristic. Ina particular embodiment the following Fact Bucket types are used:

-   -   Fact Pattern Buckets—facts which share one of the same objects        or relation    -   Site Buckets—facts which originate from the same website or        source    -   Lexical Buckets—facts with the same lexical elements (the single        words or short phrases that are translated into objects or        relations); and    -   Translation Template Buckets—the translation system (4816),        returns the template that was matched against the short sentence        in the translation of a specific fact. The translation template        captures information about the sentence structure used and        sometimes additional information such as the types of object and        some of the words contained in the sentence.

For example the fact “[peaches geldof] [is the daughter of] [bobgeldof]” extracted from the sentence “Peaches Geldof is the daughter ofBob Geldof” sourced from Wikipedia gets added to the following 8buckets:

-   -   Fact Pattern Buckets        -   peaches geldof-object        -   bob geldof-object        -   is the daughter of-relation    -   Site Buckets        -   en.wikipedia.org    -   Lexical Buckets        -   “Peaches Geldof”-object        -   “Bob Geldof”-object        -   “is the daughter of”-relation    -   Translation Template Buckets        -   translation: [“fact2@trueknowledge.com”]

According to a particular embodiment, an additional type of Fact Bucketmight be an “Author Bucket” that includes facts originating with thesame individual, i.e., the individual who wrote, produced, created,recorded, or otherwise generated the content from which the facts inthat bucket were derived. Such a bucket type would, for example,facilitate analysis of a publication site, e.g., Wikipedia, at a finergranularity, or allow the grouping of a particular journalist's columnsacross multiple news sites.

According to various embodiments, each of the Fact Buckets has areliability associated with it. A particular embodiment isnon-probabilistic, in which the reliability of a bucket can have one ofthree values: unreliable (Black), unknown (Grey), or reliable (White).The Buckets fill up every N facts that are added (5012). When a bucketfills up, it is evaluated (5014, 5102); this involves re-evaluating itspast M facts (5106). The proportion of the M facts that are true orfalse with respect to the Knowledge System governs whether the bucket isblack, white, or grey listed. If the reliability of a bucket changes(5104), it could result in the reliability of the re-evaluated factschanging.

In a particular embodiment, the bucket size, N, is 20 facts, and thelast 100 facts from that bucket are re-evaluated (M). If >3% and >3absolute facts are false, the bucket is black listed. If the bucket isnot black listed and >5 absolute facts are true and <1% of the facts arefalse, the bucket is white listed. If the bucket is not black or whitelisted it is grey listed.

A number of bucket types are classified as required buckets (used tojudge a fact reliable). Required bucket types define the conditions afact must fulfil to be considered reliable. A reliable fact must be in awhite-listed bucket for every type of required bucket. In a particularembodiment, Fact Pattern Buckets and Site Buckets are required buckets.This means that for a fact to be considered reliable it must match awhite-listed fact pattern and be sourced from a white-listed site. Forexample, if the “en.wikipedia.org” site bucket and the “is the daughterof-relation” fact pattern bucket are white listed, the fact from theprevious example will be considered reliable.

Although the Quarantine process can continue in complete automation,manual intervention may be employed to increase both precision andrecall. For example, facts can be manually evaluated or filtered and thereliability of buckets can be manually set.

7.3 Fact Extraction from Question and Answer Pairs

With the proliferation of Q&A websites (e.g., http://answer.yahoo.comand http://wiki.answers.com) designed to answer questions submitted byusers either by crowd sourcing from a community or with editorialcontent, the volume of question and answer pairs available on theinternet is significant and growing. According to some embodiments, anNLP System implemented as described herein can be augmented to takeadvantage of this additional information.

According to various embodiments, the Translation API described hereinand in U.S. Patent Publication No. US-2009-0192968-A1 (incorporated byreference above) may be used to translate the question part of the Q&Apair into a fact, partial fact, or entity list. The translated questionprovides additional information when it comes to extracting facts fromthe answer.

According to a specific embodiment, there are four situations eachrequiring a different approach when extracting facts from a Q&A pair:

-   -   Situation 1. The question has been understood by the Knowledge        System and a yes/no answer is expected in response.    -   Situation 2. The question has been understood by the Knowledge        System and an entity is expected in response.    -   Situation 3. The question has been understood by the Knowledge        System and a complex response is expected.    -   Situation 4. The question has not been understood by the        Knowledge System.

In Situation 1, the question is translated into a fact, e.g. “Did GeorgeLucas direct Star Wars” would be translated into “[george lucas][directed] [star wars]”. The polarity of the fact then needs to beextracted from the answer. In a particular embodiment, this is done bysearching the first sentence of the answer for positive or negativeterms. If the first sentence contains some positive but no negativeterms, e.g. “Yes, George Lucas did direct Star Wars”, the fact isconsidered positive (or vice versa). If no positive or negative termsappear in the first sentence, or if a mixture of terms is found, thefact is considered ambiguous and is not added to the quarantine.

In Situation 2, the question requires an entity to fill an unknown gapof knowledge, e.g. “Who directed Star Wars” would be translated to“[unknown object] [directed] [star wars]”. In this case the unknownentity is extracted from the answer. In a particular embodiment, theentity is searched for in the first sentence. Often the class of theentity will be implied. In the above example the class of [human being]is implied as it is consistent with directing a film. According tospecific embodiments, entities may searched for in two ways:

-   -   First, if the sentence is very short (e.g. “George Lucas.”), the        whole sentence is considered to be the name of the entity, and a        matching object is searched for.    -   Second, we mark-up the sentence, e.g., as outlined in the        description of the NLP System. If an entity matching the        required class is marked up it is considered to be the answer to        the question, e.g. “It was [george lucas] who directed [star        wars].”

In Situation 3, the question either cannot be answered or the answer istoo complex for a simple knowledge representation, e.g. “Why did TonyBlair invade Iraq?”. Although the answer to the asked question cannot beextracted, other facts can still be extracted from the answer. In theexample given, the question gives prior information that “Tony Blair”and “Iraq” are likely to be mentioned. In a particular embodiment, thisadditional information is used in the NLP System's entity resolutionstep (4814). If the sentence “Blair went to Egypt in 2001” appears inthe answer, the term “Blair” is ambiguous, however as we have priorinformation that the entity “Tony Blair” is likely to be referred to,the sentence can be marked up as follows: “[tony blair] went to Egypt in2001”. Note that marking up entities occurring in a question throughoutan answer can be applied in any situation where a question isunderstood.

In Situation 4, no information can be gained from the question.

Once information has been extracted from the question and answer pair,as described above, and the answer has been marked up with anyadditional information that can be gained from the question, it may betreated as a stand-alone document and processed by the NLP System asdescribed. In a particular embodiment, all facts extracted from questionand answer pairs are processed by the quarantine as standard facts.

It will be understood by those of skill in the art that embodiments ofthe present invention may be implemented in any of a wide variety ofcomputing contexts. Embodiments of the invention may also be practicedin a wide variety of network environments. In addition, the computerprogram instructions with which embodiments of the invention areimplemented may be stored in any type of non-transitorycomputer-readable storage media, and may be executed according to avariety of computing models including a client/server model, apeer-to-peer model, on a stand-alone computing device, or according to adistributed computing model in which various of the functionalitiesdescribed herein may be effected or employed at different locations withmultiple computing devices. In one class of implementations, theextraction of structured knowledge and reliability determinationsdescribed herein are effected in the context of knowledge representationsystem as depicted, for example, in FIG. 3. However, it should beunderstood that these functionalities may be implemented outside of(e.g., in parallel with) such a system. Other variations will beapparent to those of skill in the art.

While the invention has been particularly shown and described withreference to specific embodiments thereof, it will be understood bythose skilled in the art that changes in the form and details of thedisclosed embodiments may be made without departing from the spirit orscope of the invention. In addition, although various advantages,aspects, and objects of the present invention have been discussed hereinwith reference to various embodiments, it will be understood that thescope of the invention should not be limited by reference to suchadvantages, aspects, and objects. Rather, the scope of the inventionshould be determined with reference to the appended claims.

What is claimed is:
 1. A computer-implemented method for adding facts toa knowledge base, comprising: identifying a question and a correspondinganswer in unstructured text using one or more computing devices; usingthe one or more computing devices, identifying from the question a knownentity, an unknown entity, and a relationship between the known entityand the unknown entity; using the one or more computing devices,identifying in the knowledge base a first known entity objectrepresenting the known entity and a relation object representing therelationship; generating a first fact triple using the one or morecomputing devices, the first fact triple including the first knownentity, the relation object, and an unknown entity object representingthe unknown entity; identifying information in the answer correspondingto the unknown entity using the one or more computing devices;identifying a second known entity object in the knowledge base using theone or more computing devices, the second known entity objectcorresponding to the information identified in the answer; replacing theunknown entity object of the first fact triple with the second knownentity object to generate a second fact triple using the one or morecomputing devices adding the second fact triple to the knowledge baseusing the one or more computing devices.
 2. The method of claim 1,wherein identifying the unknown entity, the relationship, and the knownentity includes natural language processing of the unstructured textusing one or more of sentence splitting, sentence markup, part-of-speechtagging, sentence parsing, or sentence transformation.
 3. The method ofclaim 1, wherein identifying the information in the answer includesidentifying text in the answer corresponding to one of a plurality ofentity types.
 4. The method of claim 3, wherein identifying the secondknown entity object includes comparing the text in the answer to one ormore text strings associated with each of a plurality of entity objectsin the knowledge base, the plurality of entity objects including thesecond known entity object.
 5. The method of claim 1, further comprisingretrieving a web page including the unstructured text, whereinidentifying the question and the answer in the unstructured textincludes natural language processing of the unstructured text.
 6. Themethod of claim 1, further comprising determining a reliability scorefor the second fact triple, wherein adding the second fact triple to theknowledge base is based on the reliability score.
 7. The method of claim1, further comprising: identifying a second question and a second answercorresponding to the second question from second unstructured text;determining that the second question requires a complex response notsuited for representation as a fact triple; and disambiguating one ormore entities identified in the second question and referenced in thesecond answer.
 8. The method of claim 1, further comprising: identifyinga second question and a second answer corresponding to the secondquestion in second unstructured text; generating a third fact triplefrom the second question; and determining a polarity of the third facttriple from the answer.
 9. The method of claim 1, further comprising:receiving a user query with the one or more computing devices, the userquery corresponding to a natural language question entered on a clientdevice; retrieving the second fact triple from the knowledge base usingthe one or more computing devices; generating, using the one or morecomputing devices, a natural language answer using the second facttriple, the natural language answer being responsive to the naturallanguage question; and transmitting, using the one or more computingdevices, the natural language answer to the client device.
 10. A systemfor adding facts to a knowledge base, comprising one or more computingdevices configured to: identify a question and a corresponding answer inunstructured text; determine that the question is a yes-or-no question;identify from the question a first entity, a second entity, and arelationship between the first entity and the second entity; identify inthe knowledge base a first known entity object representing the firstentity, a second known entity object representing the second entity, anda relation object representing the relationship; generate a fact triplefrom the question, the fact triple including the first entity object,the second entity object, and the relation object; determine a polarityof the fact triple from the answer; and add the fact triple to theknowledge base.
 11. The system of claim 10, wherein the one or morecomputing devices are configured to identify the relationship and thefirst and second entities using natural language processing of theunstructured text that includes one or more of sentence splitting,sentence markup, part-of-speech tagging, sentence parsing, or sentencetransformation.
 12. The system of claim 10, wherein the one or morecomputing devices are configured to determine a polarity of the facttriple from the answer by identifying positive or negative terms in theanswer.
 13. The system of claim 10, wherein the one or more computingdevices are further configured to retrieve a web page including theunstructured text, and wherein the one or more computing devices areconfigured to identify the question and the answer in the unstructuredtext using natural language processing of the unstructured text.
 14. Thesystem of claim 10, wherein the one or more computing devices arefurther configured to determine a reliability score for the fact triple,and wherein the one or more computing devices are configured to add thefact triple to the knowledge base based on the reliability score. 15.The system of claim 10, wherein the one or more computing devices arefurther configured to: identify a second question and a second answercorresponding to the second question from second unstructured text;determine that the second question requires a complex response notsuited for representation as a fact triple; and disambiguate one or moreentities identified in the second question and referenced in the secondanswer.
 16. The system of claim 10, wherein the one or more computingdevices are further configured to: identify a second question and asecond answer corresponding to the second question from secondunstructured text; generate a second fact triple from the question, thesecond fact triple including an unknown entity object representing anunknown entity; identify information in the second answer correspondingto the unknown entity; identify a first known entity objectcorresponding to the information identified in the second answer; andreplace the unknown entity object of the second fact triple with thefirst known entity object to generate a third fact triple.
 17. Thesystem of claim 16, wherein the one or more computing devices areconfigured to identify the information in the second answer byidentifying text in the second answer corresponding to one of aplurality of entity types.
 18. The system of claim 17, wherein the oneor more computing devices are configured to identify the first knownentity object by comparing the text in the second answer to one or moretext strings associated with each of a plurality of entity objects inthe knowledge base, the plurality of entity objects including the firstknown entity object.
 19. The system of claim 10, wherein the one or morecomputing devices are further configured to: receive a user querycorresponding to a natural language question entered on a client device;retrieve the second fact triple from the knowledge base; generate anatural language answer using the second fact triple, the naturallanguage answer being responsive to the natural language question; andtransmit the natural language answer to the client device.
 20. Acomputer program product for adding facts to a knowledge base,comprising one or more computer-readable media having computer programinstructions stored therein, the computer program instructions beingconfigured such that, when executed by one or more computing devices,the computer program instructions cause the one or more computingdevices to: identify a question and a corresponding answer inunstructured text; identify from the question a known entity, an unknownentity, and a first relationship between the known entity and theunknown entity; identify in the knowledge base a first known entityobject representing the known entity and a first relation objectrepresenting the relationship; generate a first fact triple, the firstfact triple including the first known entity object, the relationobject, and an unknown entity object representing the unknown entity;identify information in the answer corresponding to the unknown entity;identify a second known entity object in the knowledge base, the secondknown entity object corresponding to the information identified in theanswer; replace the unknown entity object of the first fact triple withthe second known entity object to generate a second fact triple; add thesecond fact triple to the knowledge base; identify a second question anda second answer corresponding to the second question in secondunstructured text; determine that the second question is a yes-or-noquestion; identify from the second question a third entity, a fourthentity, and a second relationship between the third entity and thefourth entity; identify in the knowledge base a third entity objectrepresenting the third entity, a fourth entity object representing thefourth entity, and a second relation object representing the secondrelationship; generate a third fact triple from the second question, thethird fact triple including the third entity object, the fourth entityobject, and the second relation object representing the secondrelationship; determine a polarity of the third fact triple from theanswer; and add the third fact triple to the knowledge base.